Different typed robot system having edge server and cloud server and control method thereof

Abstract

The present invention relates to a cloud-based robot control method for controlling a plurality of robots placed in a plurality of arbitrarily divided spaces, comprising: a step of generating a control base model applicable to the plurality of robots on a cloud server; a step of distributing the control base model to an edge server assigned to each space; a step of upgrading the control base model according to the plurality of robots in the space on the edge server; a step of directly transmitting the upgraded control model from the edge server to another edge server; and a step of controlling the plurality of robots on the edge server using the upgraded control model. Accordingly, the present invention enables support for heterogeneous robots and heterogeneous services by sharing a deep learning model between edge servers. Furthermore, it is possible to provide optimized services by tuning the base deep learning model from the cloud server into a customized deep learning model suitable for each robot on the edge server, and upgrading it into an adaptive deep learning model suitable for the service provided by each robot.

Description

Technology Field

[0001] The present invention provides a robot system comprising a plurality of edge servers that control heterogeneous robots and a cloud server that performs transmission and reception. Specifically, the present invention relates to a method for controlling a heterogeneous robot system utilizing a cloud server that provides a base model. Background Technology

[0002] Robots were developed for industrial use and have played a role in factory automation. Recently, the fields of robot application have expanded further, with the development of medical and aerospace robots, as well as household robots for general use. Among these robots, those capable of driving under their own power are called artificial intelligence robots.

[0003] With the increasing use of robots, there is a growing demand for robots capable of providing diverse information, entertainment, and services beyond the repetitive performance of simple functions.

[0004] Accordingly, various robots are being developed to provide convenience to people by being deployed in homes, restaurants, stores, public places, etc.

[0005] In addition, heterogeneous robot systems are being developed in which different types of robots are deployed in a single defined space to perform their respective tasks.

[0006] Such a heterogeneous robot system can be defined as the convergence of Information and Communication Technologies (ICT) and existing service industries.

[0007] For example, a heterogeneous robot system is defined as a system for processing and communicating regarding various events in the space, based on technologies such as the Internet of Things, Big Data, Cloud Computing, and Cyber-Physical Systems (CPS).

[0008] Cloud computing utilizes internet technology to provide virtualized information and communication technology resources as a service.

[0009] Users can use information and communication technology resources (e.g., servers, storage, networks, and software, etc.) as needed through cloud computing.

[0010] In addition, users can receive real-time service scalability support based on the load of services provided through cloud computing, and pay for the provision of services.

[0011] When a cloud server performs control of numerous heterogeneous robots through cloud computing, an edge server may be additionally included for each space for the convenience of control or communication.

[0012] In this way, a robot system has been developed that includes an edge server controlling multiple robots at close range and a cloud server transmitting and receiving with multiple edge servers.

[0013] Such robot systems generate a very large amount of data traffic between cloud servers and edge servers.

[0014] In the case of U.S. Patent Publication No. 2020-0050951, it is disclosed that one of the edge nodes generates a specification of a machine learning model and distributes the specification to multiple edge nodes.

[0015] That is, the model is updated by performing machine learning on each other through parameter exchange between multiple edge nodes.

[0016] However, when parameters are exchanged between edge nodes in this way, there is a disadvantage that it is difficult to support heterogeneous robots or heterogeneous services.

[0017] In addition, U.S. Public Patent No. 2020-0079898 discloses that an edge server performs a specific action through inference using a machine learning model, evaluates the accuracy of the specific action, and if the specific action is inaccurate, the edge server collects data regarding the specific action and transmits it to a cloud server, and the cloud server learns a new machine learning model using the collected data.

[0018] However, in such cases, a large amount of data must be transmitted and received between the edge server and the cloud server, which incurs significant time and cost, and it is very inefficient on the edge server because a new machine learning model must be trained.

[0019] Meanwhile, Korean published patent No. 2020-0063340 discloses that an edge server processes data requiring real-time processing, and a central processing server processes advanced machine learning and large-scale data. At this time, the central processing server converts a general-purpose deep learning neural network into a deep learning neural network of a product similar to a specific product requiring learning, and retrains the converted deep learning neural network with a learning data set of the specific product.

[0020] However, in such cases, there is a disadvantage in that retraining requires a significant amount of time and processing. Prior art literature

[0021] U.S. Published Patent No. 2020-0050951, Date of Publication February 13, 2020 U.S. Published Patent No. 2020-0079898, Date of Publication March 14, 2019 Korea Published Patent No. 2020-0063340, Date of Publication June 5, 2020 The problem to be solved

[0022] The first objective of the present invention is to provide a service for efficient intelligence augmentation by sharing a deep learning model for controlling heterogeneous robots between edge servers.

[0023] To this end, the second objective of the present invention is to provide a control method utilizing an edge server that tunes a base deep learning model into a customized deep learning model suitable for each robot, and upgrades it into an adaptive deep learning model suitable for the service provided by each robot.

[0024] In addition, the third objective of the present invention is to provide a control method that can minimize processing data passing through a cloud server by selecting and directly sharing a deep learning model necessary for intelligence augmentation between edge servers, and by expanding the existing deep learning model on the edge server by replacing it with the shared deep learning model. means of solving the problem

[0026] The present invention provides a cloud-based robot system comprising: a plurality of robots disposed in a plurality of arbitrarily divided spaces; a cloud server that generates and distributes a control base model applicable to the plurality of robots; and a plurality of edge servers each assigned to a space, transmits and receives with the cloud server, receives the control base model, upgrades it according to the plurality of robots in the space, and controls the plurality of robots with the upgraded control model, wherein the edge servers directly transmit the upgraded control model to other edge servers.

[0027] The above plurality of robots may include different types of robots.

[0028] The above control base model can be packaged with respective control models for multiple functions of different types of the robot.

[0029] The edge server receives the control base model and can execute the control base model by tuning it according to the type of robot controlled by the edge server.

[0030] The edge server can generate an upgraded control model by performing deep learning error learning on the tuned control base model.

[0031] The cloud server can obtain information about the upgraded control model, select other edge servers to which the upgraded control model is applied, and directly transmit the upgraded control model between the edge servers.

[0032] The cloud server can select the edge server containing the robot to which the upgraded control model is applied and transmit the upgraded control model.

[0033] The edge server can generate the upgraded control model by performing deep learning error learning when the error values ​​exceeding a threshold are greater than a predetermined number while executing the control base model.

[0034] The cloud server above can classify multiple edge servers into multiple groups and manage them.

[0035] One group can be located within a predetermined distance or a predetermined response time.

[0036] Meanwhile, another embodiment of the present invention is a cloud-based robot control method for controlling a plurality of robots placed in a plurality of arbitrarily divided spaces, comprising: a step of generating a control base model applicable to the plurality of robots in a cloud server; a step of distributing the control base model to an edge server assigned to each space; a step of upgrading the control base model according to the plurality of robots in the space in the edge server; a step of directly transmitting the upgraded control model from the edge server to another edge server; and a step of controlling the plurality of robots in the edge server with the upgraded control model.

[0037] The above plurality of robots may include different types of robots.

[0038] The above control base model generation step can generate and package respective control models for multiple functions of different types of robots.

[0039] The method may further include the step of receiving the control base model from the edge server and tuning the control base model according to the type of robot controlled by the edge server.

[0040] The above-mentioned upgrading step can generate an upgraded control model by performing deep learning error learning on the tuned control base model.

[0041] The method may include the steps of obtaining information about the upgraded control model from the cloud server, selecting another edge server to which the upgraded control model is applied, transmitting information of the other edge server selected as the upgraded edge server, and directly transmitting the upgraded control model from the upgraded edge server to the other edge server selected.

[0042] The step of selecting the other edge server mentioned above allows the cloud server to select an edge server that includes a robot to which the upgraded control model is applied.

[0043] The above upgrade step can generate the upgraded control model by performing the deep learning error learning when the error value exceeding a threshold is greater than a predetermined number while running the control base model on the edge server.

[0044] The method may further include the step of classifying and grouping a plurality of the edge servers from the cloud server.

[0045] One group can be located within a predetermined distance or a predetermined response time. Effects of the invention

[0046] Through the above solution, the present invention enables support for heterogeneous robots and heterogeneous services by sharing deep learning models between edge servers.

[0047] In addition, it is possible to provide optimized services by tuning a base deep learning model from a cloud server into a customized deep learning model suitable for each robot on an edge server, and upgrading it into an adaptive deep learning model suitable for the services provided by each robot.

[0048] Between edge servers, deep learning models necessary for intelligence augmentation are selected and shared directly, and by expanding by replacing existing deep learning models on edge servers with the shared deep learning models, processing data passing through cloud servers can be minimized.

[0049] In addition, efficient knowledge augmentation is possible by updating and training an existing deep learning model with a deep learning model from another edge server shared based on a base model from a cloud server. Brief explanation of the drawing

[0050] FIG. 1 is a schematic diagram illustrating a cloud and edge-based environment according to one embodiment of the present invention. Figure 2 is a conceptual diagram showing the zone classification of the cloud server of the present invention. FIG. 3 is a detailed diagram illustrating a heterogeneous robot system based on one edge server of the present invention. Figure 4 is a configuration diagram according to an application example of the cloud server or edge server of Figure 3. FIG. 5 is a conceptual diagram illustrating the modeling of a cloud and edge server according to an embodiment of the present invention. Figure 6 is an overall flowchart of the robot cloud system of the present invention. Figure 7 is a detailed flowchart showing how one edge server enters the system of Figure 6. Figure 8 is a flowchart showing one edge server of Figure 6 upgrading the base model. FIG. 9 is a flowchart illustrating a method for sharing an upgrade model of another edge server according to an embodiment of the present invention. Specific details for implementing the invention

[0051] Expressions designating directions such as “Front (F) / Rear (R) / Left (Le) / Right (Ri) / Up (U) / Down (D)” mentioned below are defined according to their indications in the drawings; however, this is merely for the purpose of explaining the invention so that it can be clearly understood, and it goes without saying that each direction may be defined differently depending on where the reference is placed.

[0052] The use of terms such as 'first, second,' etc., preceding the components mentioned below is intended solely to avoid confusion regarding the components being referred to, and is unrelated to the order, importance, or master-subordinate relationship between the components. For example, an invention including only the second component without the first component can be implemented.

[0053] In the drawings, the thickness or size of each component is exaggerated, omitted, or depicted schematically for the convenience and clarity of explanation. Additionally, the size and area of ​​each component do not fully reflect their actual size or area.

[0054] In addition, angles and directions mentioned in the process of describing the structure of the present invention are based on those described in the drawings. In the description of the structure in the specification, if the reference point and positional relationship for an angle are not clearly mentioned, the relevant drawings should be referenced.

[0055] FIG. 1 is a schematic diagram for explaining a cloud and edge-based environment according to an embodiment of the present invention, FIG. 2 is a conceptual diagram showing the zone classification of a cloud server of the present invention, and FIG. 3 is a detailed diagram for explaining a heterogeneous robot system based on one edge server of the present invention.

[0056] Referring to FIG. 1, a cloud-based robot system according to one embodiment of the present invention may include a cloud server (10), a plurality of edge servers (20), and a plurality of robots (30).

[0057] The cloud-based robot (30) system manages and controls multiple robots (30) distributed in various spaces spaced apart from each other.

[0058] At this time, each space is functionally or geographically separated, and each edge server (20) is set up to control a plurality of robots (30) placed within each space.

[0059] In each space, robots of the same type (30) may be placed, but, conversely, robots of different types (30) may be placed.

[0060] For example, in space 1 (A1), an edge server 1 (21) may be set up, and one order robot 1 (313) and two guide robots (311, 312) may be placed therein. Meanwhile, in space 2, an edge server 2 (22) may be set up, and two delivery robots (321, 323) and one guide robot (322) may be placed therein. Meanwhile, in space 3, an edge server n (23) may be set up, and an entertaining robot (331) and a guide robot (332) may be placed therein.

[0061] The heterogeneous or homogeneous robots (30) placed in each space transmit and receive with the edge server (20) set up in the space and can provide assigned services by interacting with the user in a home or workplace, etc.

[0062] Preferably, a cloud robot system according to one embodiment of the present invention includes a plurality of heterogeneous robots (30) and each edge server (20) set up in each space, and a cloud server (10) that transmits and receives with each edge server (20) and provides integrated management.

[0063] The edge server (20) can monitor and control the status of multiple robots (30) remotely, so the cloud robot system can provide more effective services by using multiple robots (30).

[0064] A plurality of heterogeneous robots (30), edge servers (20), and cloud servers (10) may communicate with each other by having communication means (not shown) that support one or more communication standards.

[0065] For example, multiple heterogeneous robots (30), edge servers (20), and cloud servers (10) can be implemented to communicate wirelessly using wireless communication technologies such as IEEE 802.11 WLAN, IEEE 802.15 WPAN, UWB, Wi-Fi, Zigbee, Z-wave, Blue-Tooth, etc. The robot (30) may vary depending on the communication method of the other device or server (2) to which it wishes to communicate.

[0066] In particular, multiple robots (30) can implement wireless communication with other robots (30) and / or edge servers (20) and cloud servers (10) via a 5G network. When a robot (30) communicates wirelessly via a 5G network, it can perform ultra-low latency / ultra-high capacity data transmission networks.

[0067] More specifically, 5G networks are communication technologies that provide transmission speeds of tens of Gbps in the wireless section. They enable the transmission of ultra-low latency, ultra-high capacity, and ultra-realistic data at speeds exceeding Gbps and ultra-low latency in the millisecond range across multiple modules to meet service-specific quality requirements. Such 5G networks can provide network quality equivalent to high-speed wired networks while simultaneously offering the advantages of being wireless.

[0068] This 5G network can be controlled to provide various services optimized for each heterogeneous robot (30) between the robot (30) and the edge server (20) and between the edge server (20) and the cloud server (10) in a cloud machine learning / deep learning-based robot system.

[0069] Additionally, multiple robots (30) and edge servers (20) can communicate using MQTT (Message Queueing Telemetry Transport) and HTTP (HyperText Transfer Protocol), but are not limited thereto.

[0070] In some cases, the multiple robots (30), edge server (20), and cloud server (10) support two or more communication standards, and the optimal communication standard can be used depending on the type of communication data and the type of device participating in the communication.

[0071] The user can check information about the robots (30) within the robot (30) system while transmitting and receiving with the edge server (20).

[0072] In this specification, 'user' refers to a person who uses a service through a plurality of robots (30), and may include individual customers who purchase or rent the robots (30) and use them at a workplace, etc., as well as managers and employees of a company that provides services to employees or customers using the robots (30), and customers who use the services provided by such company. Accordingly, 'user' may include individual customers (Business to Consumer: B2C) and corporate customers (Business to Business: B2B).

[0073] The cloud server (10) designs / creates a model for controlling multiple physically distributed robots (30) and distributes the model to edge servers (20) set up in each space.

[0074] Specifically, the cloud server (10) generates a model or engine in the cloud (10). In the environment of a heterogeneous robot system, a general-purpose model or general-purpose engine applied to a heterogeneous robot (30) is a base model and is not designed as a model specific to a particular situation.

[0075] The cloud server (10) can form and package multiple models or engines that are functionally or structurally distinct for multiple heterogeneous robots (30), and can perform complex machine learning / deep learning that is difficult to implement on the edge server (20).

[0076] Machine learning is a field of artificial intelligence that refers to the technology of creating models capable of generalizing, classifying, and evaluating vast amounts of data, based on algorithms and techniques that enable computers to learn autonomously.

[0077] By creating such a base model and distributing it to multiple edge servers (20), control can be performed on multiple heterogeneous robots (30) that are physically separated.

[0078] In addition, the cloud server (10) can group multiple edge servers (20) by zone, and the edge servers (20) grouped in this way can be managed by group.

[0079] At this time, the grouped zones can be configured based on the physical distance or response time from the cloud server (10) to the edge server (20).

[0080] As shown in FIG. 2, a plurality of edge servers (21, 22) existing within a first distance (d1) or a first response time from a cloud server (10) can be classified into a first zone (z1), a plurality of edge servers (23, 24, 25) existing within a second distance (d2) greater than the first distance (d1) or a second response time longer than the first response time can be classified into a second zone (z2), and other edge servers (26, 27) can be classified into a third zone (z3) by a third distance (d3) or a third response time.

[0081] At this time, the distances between the first distance (d1), the second distance (d2), and the third distance (d3) may be the same, but are not limited thereto.

[0082] In addition, the difference between the first and third response times may be the same, but is not limited thereto. In this case, the response time may be defined as the number of network nodes until the packet arrives or the response time obtained via ping, but is not limited thereto.

[0083] In this way, when multiple edge servers (20) are classified by zone, they can be usefully applied to the modeling, distribution, scheduling, etc. of each edge server (20) in the cloud server (10).

[0084] The edge server (20) is set up in each space as shown in FIG. 3, and controls each robot (30) by tuning and upgrading the base model into an adaptive model to suit a plurality of heterogeneous or homogeneous robots (30) placed in the space.

[0085] A general edge server (20) is a technology that processes data in the edge area where data is created, deviating from the method of processing data in the cloud server (10). Data requiring real-time processing is processed at the edge server (20), and if necessary, it is implemented in a way that communicates with the central cloud as a secondary task.

[0086] However, in the present invention, a cloud server (10) performs a base deep learning modeling, which is a general-purpose model or engine for heterogeneous robots (30) and services, and an edge server (20) receives the base model and tunes it to be suitable for the space to provide a service as an adaptive deep learning model.

[0087] Specifically, the edge server (20) utilizes training data of the space previously collected from various multiple robots (30) to execute a base deep learning model from the cloud server (10) by upgrading it in a customized manner suitable for each robot (30) and service.

[0088] At this time, the edge server (20) can take into account various environmental information about the space and can be upgraded to an optimal customized deep learning model.

[0089] Additionally, when providing a service, the edge server (20) can transmit feedback (errors, performance information, etc.) from multiple heterogeneous robots (30) to the cloud server (10) and share the upgraded model with other edge servers (20). Accordingly, other edge servers (20) that receive the shared model can remove the existing model, update it with the shared model, and tune and execute it to suit the robots (30) in the space.

[0090] Each edge server (20) is placed in a separate virtual space or a physically separated space (A1) as shown in FIG. 3, and controls a plurality of robots (30) that are placed in the space (A1) and provide services, and the services provided by the robots (30).

[0091] For example, as shown in FIG. 3, an edge server (21) can control a plurality of robots (30: 31, 32, 33, 34, 35) that provide services within a smart restaurant, and the plurality of robots (30) may include a guide robot, an ordering robot, a delivery robot, an entertaining robot, a cleaning robot or a cooking robot, a barista robot, etc.

[0092] In this way, multiple heterogeneous robots (30) placed in the space (A1) are dependent on the edge server (21) and provide services while transmitting and receiving data.

[0093] These cloud servers (10) or edge servers (21) can generally have similar configurations.

[0094] FIG. 4 is a configuration diagram according to one application example of the cloud server (10) or edge server (20) of FIG. 3.

[0095] Referring to FIG. 4, the edge server (20) or cloud server (10) may include memory, a computing data collection database (110), a control unit (100), and a communication unit (120).

[0096] Specifically, the cloud server (10) or edge server (20) may be implemented as at least one computing device composed of a memory and a processor that reads a program stored in the memory and performs a specific function.

[0097] First, the memory (110) can store and manage data acquired based on IoT technology from multiple homogeneous or heterogeneous robots (30) in the space in the case of the edge server (20), and can store a base deep learning model received from the cloud server (10).

[0098] In the case of a cloud server (10), it can store an algorithm capable of performing processing for base deep learning.

[0099] Additionally, the memory (110) can store one or more of the application programs, data, and instructions required for functional operation according to an embodiment of the present invention.

[0100] This memory (110) may be various storage devices such as ROM, RAM, EPROM, flash drive, hard drive, etc., and may also be web storage that performs the storage function of the memory (110) on the internet.

[0101] Additionally, in some embodiments, software components stored in memory (110) may include an operating system, a communication module (or set of instructions), a contact / motion module (or set of instructions), a graphics module (or set of instructions), a text input module (or set of instructions), a satellite positioning system (GPS) module (or set of instructions), and applications (or sets of instructions).

[0102] The processor (100) can control and drive the overall operation of multiple robots (30) in the case of an edge server (20).

[0103] Such a processor (100) can be implemented using at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and other electrical units for performing functions.

[0104] The processor (100) of the edge server (20) can analyze various data obtained from multiple robots (30) and perform appropriate processing, and can perform machine learning / deep learning on its own to perform learning for upgrading the base deep learning model.

[0105] The edge server (20) can transmit upgraded data to the robot (30) after performing machine learning / deep learning based on data received from the robot (30), data input by the user, etc. Accordingly, the control program equipped by the robot (30) can be updated.

[0106] The processor (100) of the edge server (20) can analyze the data obtained from a plurality of robots (30) respectively and apply each analyzed data to the corresponding model or engine.

[0107] The processor (100) of the cloud server (10) may be equipped with a more advanced processor capable of performing machine learning / deep learning.

[0108] Meanwhile, the cloud server (10) or edge server (20) may include a communication unit (120), and as previously described, various types of communication are possible depending on the communication method required by the environment. In particular, communication between the robot (30) and the edge server (20), between the edge server (20) and the cloud server (10), and between the edge server (20) and the edge server (20) is possible using a 5G network, and the algorithm of a large-capacity deep learning modeling itself can be transmitted and received.

[0109] Meanwhile, various heterogeneous or homogeneous robots (30) placed in each space can transmit space, object, and usage-related data to the edge server (20).

[0110] Here, the data may be data related to the recognition of space and object recognized by the robot (30), or image data related to space and object acquired by the image acquisition unit.

[0111] According to an embodiment, the robot (30) and the edge server (20) may include an artificial neural network (ANN) in the form of software or hardware that is trained to recognize at least one of the attributes of an object, such as a user, voice, attributes of space, and obstacles.

[0112] According to one embodiment of the present invention, the cloud server (10) and the edge server (20) may include a deep neural network (DNN), such as a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), or a Deep Belief Network (DBN), which is trained by deep learning. For example, the processor of the edge server (20) may be equipped with a deep neural network structure (DNN), such as a Convolutional Neural Network (CNN).

[0113] Additionally, usage-related data is data obtained through the use of the robot (30), and may include usage history data, detection signals obtained from the sensor unit (110), etc.

[0114] The robot (30) can receive recognition input data and recognize the attributes of people, objects, and spaces included in the input data, and output the result.

[0115] In addition, the above-mentioned learned deep neural network structure (DNN) receives input data for recognition, analyzes and learns data related to the use of the robot (30), and can recognize usage patterns, usage environments, etc.

[0116] Meanwhile, data related to space, object, and usage can be transmitted to the edge server (20) through the communication unit of the robot (30).

[0117] The edge server (20) can train a deep neural network (DNN) based on the received data, and then transmit the updated deep neural network (DNN) structure data to the corresponding robot (30) to update it.

[0118] A robot (30) according to one embodiment of the present invention may be a mobile robot (30) that travels through each space and performs a set service or mission.

[0119] Such a robot (30) may include a control unit that controls overall operation, a storage unit that stores various data, and a communication unit that transmits and receives data with other devices such as an edge server (20).

[0120] The control unit can control the communication unit, various sensors, etc. within the robot (30) to control the overall operation of the robot (30).

[0121] The communication unit includes at least one communication module to enable the artificial intelligence robot (30) to be connected to the internet or a predetermined network and to communicate with other devices.

[0122] In addition, the communication unit can process data transmission and reception between the robot (30) and the edge server (20) by connecting to a communication module provided in the edge server (20).

[0123] A robot (30) according to one embodiment of the present invention may further include a voice input unit that receives voice input from a user through a microphone.

[0124] The voice input unit may include a processing unit that converts analog sound into digital data or is connected to the processing unit, and may convert the user input voice signal into data so that it can be recognized by the control unit or the edge server (20).

[0125] Meanwhile, the control unit can control the robot (30) to perform a predetermined operation based on the voice recognition result.

[0126] Meanwhile, the robot (30) may include various modules that display predetermined information as an image or output it as sound, depending on the type.

[0127] The robot (30) may include a display that displays information corresponding to a command input, a processing result corresponding to a user's command input, an operation mode, an operation state, an error state, etc., as an image.

[0128] According to an embodiment, at least a portion of the display may be configured as a touchscreen by forming a layered structure with a touchpad. In this case, the display configured as a touchscreen may also be used as an input device capable of inputting information by user touch, in addition to being an output device.

[0129] In addition, the sound output unit can output sound, notification messages such as warning sounds, operating modes, operating statuses, and error statuses, information corresponding to user command inputs, and processing results corresponding to user command inputs, according to the control of the control unit. The sound output unit can convert electrical signals from the control unit into audio signals and output them. To this end, a speaker or the like may be provided.

[0130] According to an embodiment, the robot (30) may further include an image acquisition unit capable of capturing a predetermined range.

[0131] The image acquisition unit captures the surroundings of the artificial intelligence robot (30), the external environment, etc., and may include a camera module. Multiple such cameras may be installed in each part to improve shooting efficiency.

[0132] The image acquisition unit can capture an image for user recognition. The control unit can determine the external situation or recognize the user (target of guidance) based on the image captured and acquired by the image acquisition unit.

[0133] In addition, if the robot (30) is an artificial intelligence robot (30), the control unit can control the robot (30) to drive based on the image acquired by the image acquisition unit.

[0134] The robot (30) may further include a driving unit for movement, and the driving unit may move the main body according to the control of the control unit.

[0135] The drive unit may be positioned within the driving unit of the robot (30) and may include at least one drive wheel (not shown) that moves the main body. The drive unit may include a drive motor (not shown) connected to the drive wheel to rotate the drive wheel. The drive wheels may be provided on the left and right sides of the main body, respectively, and are referred to as the left wheel and the right wheel, respectively.

[0136] The left and right wheels may be driven by a single drive motor, but if necessary, a left-wheel drive motor to drive the left wheel and a right-wheel drive motor to drive the right wheel may be provided separately. The driving direction of the main body can be changed to the left or right by creating a difference in the rotational speeds of the left and right wheels.

[0137] Meanwhile, the robot (30) may include a sensor unit that includes sensors for sensing various data related to the operation and state of the robot (30).

[0138] The sensor unit may further include a motion detection sensor that detects the motion of the robot (30) and outputs motion information. For example, a gyro sensor, a wheel sensor, an acceleration sensor, etc., may be used as the motion detection sensor.

[0139] The sensor unit may include an obstacle detection sensor that detects obstacles, and the obstacle detection sensor may include an infrared sensor, an ultrasonic sensor, an RF sensor, a geomagnetic sensor, a PSD (Position Sensitive Device) sensor, a cliff detection sensor that detects whether there is a cliff on the floor within the driving area, a Lidar (light detection and ranging: Lidar), etc.

[0140] Meanwhile, the obstacle detection sensor detects objects, particularly obstacles, that exist in the driving (movement) direction of the artificial intelligence robot (30) and transmits obstacle information to the control unit. At this time, the control unit can control the movement of the robot (30) according to the location of the detected obstacle.

[0141] Meanwhile, the control unit can control the transmission of the operating status of the artificial intelligence robot (30) or user input, etc., to the edge server (20) through the communication unit.

[0142] As shown in FIG. 3, such a robot (30) may have a structure in which specific tasks, such as a delivery robot, an order robot, and a guide robot, are assigned to a single space, and a work section may be separately included to suit the task.

[0143] Each robot (30) can be controlled from each edge server (20) according to the assigned mission.

[0144] Hereinafter, the control of a robot (30) system including a heterogeneous robot (30) will be described in detail with reference to FIGS. 5 to 7.

[0145] FIG. 5 is a conceptual diagram illustrating the modeling of a cloud and edge server (20) according to an embodiment of the present invention, FIG. 6 is an overall flowchart of the robot (30) cloud system of the present invention, and FIG. 7 is a detailed flowchart showing how one edge server (20) enters the system of FIG. 6.

[0146] Referring to FIG. 6, a robot (30) cloud system according to one embodiment of the present invention performs model creation / distribution / sharing between a cloud server (10) and a plurality of edge servers (20) that transmit and receive data.

[0147] Specifically, the cloud server (10) creates a base model that can be universally applied to all robots (30) controlled by the cloud server (10), that is, robots (30) that are controlled by the cloud server (10) or connected to store information, or robots (30) connected to the edge server (20) (S10).

[0148] First, the cloud server (10) performs machine learning / deep learning on the processor (100) to generate a model that can be universally used for controlling multiple heterogeneous robots (30).

[0149] When a heterogeneous robot (30) controlled through an edge server (20) that transmits and receives to a cloud server (10) is a delivery robot, barista robot, ordering robot, cooking robot, guidance robot, etc. as shown in FIG. 5, an object recognition model (101), a navigation model (102), a voice recognition model (103), a location recognition model (104), an emotion recognition model (105), a control intelligence model (106), etc., can be generated as models that can be selectively utilized for each of the robots (30).

[0150] Such models (101-106) can be selectively applied to a specific robot (30), and in the case of a cloud server (10), all required models (101-106) are generated respectively in the concept of a union.

[0151] Each of these deep learning models is learned through a process in which a general-purpose packaged base model is provided to the edge server (20), and each edge server (20) is optimized by tuning / upgrading according to each environment and robot (30).

[0152] Specifically, the cloud server (10) creates a packaged base model and distributes the packaged base model to each registered edge server (20) (S10).

[0153] Meanwhile, the cloud server (10) performs grouping by registering multiple edge servers (20) and classifying them into specific zones (S20).

[0154] As shown in FIG. 7, a zone is established according to the distance or response time of each registered edge server (20), and information about the zone assigned to a plurality of edge servers (20) in the same zone and information about other edge servers (20) registered in the same zone are transmitted (S100).

[0155] At this time, when a service registration request is received from the new edge server (22), the cloud server (10) performs zone allocation for the new edge server (22) (S101).

[0156] That is, the physical location of the edge server (22) is determined through the IP address of the new edge server (22), and the response time is measured through Ping or traceroute (S102).

[0157] The cloud server (10) reads the distance from the cloud server (10) based on the response time or physical location and assigns a new edge server (20) to a specific zone (S103).

[0158] The cloud server (10) transmits information regarding zone allocation, such as the range of the zone and information about other edge servers (21, 23, etc.) registered within the zone, to the new edge server (22) (S104). Additionally, it transmits to the other edge servers (21, 23, etc.) in the zone that the new edge server (22) has been registered (S105).

[0159] Returning to Fig. 6, the cloud server (10) deploys the corresponding packaged base model to the newly registered edge server (22) (S31).

[0160] At this time, the distributed base model is a package containing a model or module related to various functions capable of managing or controlling various heterogeneous robots (30), and an application for upgrading or tuning the corresponding base model may also be distributed together when the base model is transmitted.

[0161] When the new edge server (22) receives the base model, it tunes the base model to be suitable for a plurality of heterogeneous robots (30) managed by the edge server (22) (S40).

[0162] In this tuning, as shown in FIG. 5, when the new edge server (22) controls only the delivery robot (30), the order robot (30), and the guide robot (30), the control intelligence model among the received base models is not activated, and only other models such as the object recognition model (101), the navigation model (102), the voice recognition model (103), the location recognition model (104), and the emotion recognition model (105) are activated.

[0163] In other words, tuning is defined as the process of activating only the models necessary in each area and saving the model that is not needed in an inactive state.

[0164] The new edge server (22) can reduce costs and time by reducing computational programs by selecting and activating only the models required for the heterogeneous robot (30) in the zone (A2).

[0165] For example, in the case of the first edge server (21) of FIG. 5, if a delivery robot, a barista robot, an ordering robot, a cooking robot, and a guide robot are placed within the zone (A1), all models packaged as base models can be activated.

[0166] In the case where only the barista robot, order robot, and cooking robot exist, such as in the third edge server (23), the robots do not move, so control can be carried out without the navigation model (102), location recognition model (104), etc. being activated.

[0167] Next, the edge server (20) can drive the tuned base model to control each heterogeneous robot (30), and when a predetermined data is obtained, each base model can be upgraded (S50).

[0168] The upgrade of such a base model will be explained in detail later with reference to FIGS. 8 and FIGS. 9.

[0169] When an upgrade to the base model is performed at one edge server (20), the edge server (20) transmits the upgrade information to the cloud server (10) (S61).

[0170] The cloud server (10) receives upgrade information, selects edge servers (20) that require the upgraded base model, and transmits information about the selected edge servers (20) to the edge servers (20) (S62).

[0171] The edge server (20) can share the upgraded base model with other edge servers (20) by transmitting the upgraded base model to the selected edge servers (20) (S63).

[0172] At this time, the selected edge server (20) that receives the upgraded base model can delete the previous base model, convert and update it to the upgraded base model, and control the robot (30) within the allocated area (S70).

[0173] In this way, the control system of heterogeneous robots (30) using the cloud server (10) of the present invention includes an edge server (20) for controlling robots (30) at a distance, and each edge server (20) appropriately tunes and uses a base model provided by the cloud server (10), and upgrades the base model by performing machine learning / deep learning.

[0174] In this way, the base model upgraded to be specialized for each environment at the edge server (20) can be directly transmitted to and received by other edge servers (20) that require it.

[0175] In this way, the base model upgraded is not transmitted and received through the cloud server (10), but is transmitted and received directly from the edge server (20) to another edge server (20) that is needed, so unnecessary traffic can be reduced and sufficient storage space of the cloud server (10) can be secured.

[0176] Below, the upgrade of the base model at each edge server (20) and the sharing of the upgraded base model will be explained with reference to FIGS. 8 and FIGS. 9.

[0177] FIG. 8 is a flowchart showing one edge server of FIG. 6 upgrading a base model, and FIG. 9 is a flowchart showing a method for sharing an upgrade model of another edge server according to an embodiment of the present invention.

[0178] First, referring to FIG. 8, when an edge server (20) receives a grouped base model from a cloud server (10), it tunes the base model to be suitable for a plurality of heterogeneous robots (30) controlled by the edge server (20).

[0179] For example, if all models are required, such as in the first edge server (21) of FIG. 5, they can be used without separate tuning, and tuning can be performed by disabling the control intelligence model, such as in the second edge server (22).

[0180] When applying the tuned base model to a plurality of robots (30) to perform control, a threshold value and a minimum number of data for deep learning error learning are set (S51).

[0181] Specifically, a threshold value (Eth) and a minimum number of data points (N) can be set.

[0182] The threshold value (Eth) can be defined as a criterion value for selecting values ​​classified as error values ​​among the values ​​received for each base model, and the minimum number of data (N) can be defined as the minimum number of data that allows deep learning model training.

[0183] At this time, the threshold value (Eth) and the minimum number of data points (N) can be set differently for each model.

[0184] Next, the edge server (20) controls multiple heterogeneous robots (30) by utilizing each of their respective models (S52).

[0185] At this time, the edge server (20) receives the input value transmitted to each robot (30), the output value and error value (E) output from each robot (30), respectively.

[0186] Such input values, output values, and error values ​​(E) can be received for each service or each function's progress and for each event's occurrence.

[0187] The edge server (20) compares each error value (E) for one model with a threshold value (Eth) (S53).

[0188] At this point, if the error value (E) is greater than the threshold value (Eth), it is determined to be true, saved as training data, and the number of data (n=n+1) is counted (S54).

[0189] In this way, while determining whether the error value (E) for the data is greater than the threshold (Eth) in succession, if the counted number of data does not satisfy the minimum number of data (N), the next data is read and the true / false of the error value (E) is determined again, and if the number of data with an error value greater than the threshold (Eth) satisfies the minimum number of data (N), it is determined that the deep learning model can be trained and an upgrade is performed through deep learning training of the model (S55).

[0190] At this time, the edge server (20) can be upgraded by changing the number of nodes, the number of layers, and the activation function values ​​along with the added error values, and at this time, the activation function can utilize multiple candidate sets stored in advance (S56).

[0191] For example, when applying the speech recognition model, if the error value exceeds the minimum data value, the program can be configured to minimize the error by applying a different activation function accordingly.

[0192] The model upgraded in this way can be stored in the memory (101) of the edge server (20) as version 2 (V2), and can be used to replace the previous version, version 1 (V1).

[0193] When a change is made to the upgraded model, the edge server (20) transmits the details of such change to the cloud server (10) and controls the heterogeneous robot (30) again (S57).

[0194] In this way, each edge server (20) can upgrade the base model according to its own feedback, thereby enabling the modeling of the version most optimized for the environment, and by receiving the base model from the cloud server (10) and upgrading it, very much computation is not required to generate the base model itself.

[0195] In addition, since each activation function and change option is already stored, upgrading is possible by modifying them, allowing for a relatively simple upgrade.

[0196] In this way, the base model upgraded at a specific edge server (20) can be shared with other edge servers (20) that control similar robots (30).

[0197] Specifically, referring to FIG. 9, when a model upgrade occurs at edge server A (21) (S200), edge server A (21) transmits such upgrade change details to cloud server (10) (S201).

[0198] At this time, Edge Server A (21) can notify another Edge Server (22) that an event has occurred requiring communication.

[0199] That is, since many errors occur in the model and similar errors are expected to occur in other edge servers (22), the need for a model upgrade to prevent such errors is transmitted to the cloud server (10).

[0200] Additionally, the edge server A (21) transmits the changes and the related robot (30) model name, rather than the entire upgraded base model, to the upgraded model (S202).

[0201] That is, by transmitting only information about the upgraded changes and the related robot (30) model, the cloud server (10) selects the edge server (20) that requires the application of the base model upgraded in this way based on the information (S203).

[0202] That is, the cloud server (10) can select other edge servers (22) that control the same robot (30), and can also select edge servers (22) that mainly utilize the upgraded model.

[0203] At this time, the cloud server (10) may select only the edge servers (22) within the zone to which the upgraded edge server A (21) belongs by referring to the zone-specific relevance of each edge server (20), but is not limited thereto.

[0204] In this way, the cloud server (10) selects the edge server (20) that is needed and transmits information about the corresponding edge server (22) to edge server A (21) (S204).

[0205] Edge server A (21) receives information about selected edge servers (22), such as IP addresses, and prepares to transmit the upgraded base model to the corresponding edge servers (22).

[0206] Meanwhile, the cloud server (10) plans to transmit the upgrade base model to the selected edge servers (22) (S205).

[0207] That is, each selected edge server (22) is notified that an upgrade of the corresponding base model has occurred, and is notified that the upgraded base model will be transmitted from edge server A (21).

[0208] Each selected edge server (22) prepares to receive the corresponding upgraded base model by receiving a notification from the cloud server (10).

[0209] At this time, Edge Server A (21) may notify the selected Edge Servers (22) of the deep learning model change and the related robot (30) model name, and request them to prepare for an update to the upgraded base model (S206).

[0210] Selected edge servers (22) move the currently stored deep learning base model to temp and store it temporarily (S207).

[0211] Next, the selected edge servers (22) each reply to edge server A (21) that they are ready and send an update request (S208).

[0212] Edge server A (21) distributes the upgraded deep learning model to each selected edge server (22) (S209).

[0213] When a selected edge server (22) receives an upgraded deep learning model, it stores the model and performs a trial run.

[0214] If there are no errors in the execution of the upgraded deep learning model, the temp is discarded to delete the previous deep learning base model (S210).

[0215] In this way, the deep learning base model is updated, and the version of the updated model is notified to the cloud server (10) as version 2 (V2) (S211).

[0216] At this time, the edge server A (21) is also notified of the completion of the model update (S212).

[0217] The edge server (22) selected in this way can each execute the base model received from the cloud server (10) by tuning it, and when upgrade information and selection information are received from another edge server (20), it can receive the upgraded model, update the tuned base model to the next version, and control the robots (30) in the area with the upgraded version.

[0218] In addition, each edge server (20) can upgrade the model by deep learning to the next version based on counting a predetermined number of data and determining an error, as shown in FIG. 8, even when controlling to the upgraded version, and this can also be shared with other edge servers (20).

[0219] An embodiment of the present invention can continuously upgrade by sharing intelligence between two or more edge servers (20), thereby sharing the evolved models as they evolve in their respective environments.

[0220] Therefore, since only feature points and history are stored without storing the versioned model on the cloud server (10), storage space on the cloud server (10) can be secured.

[0221] In addition, adaptive modeling is possible by performing upgrades at the edge server (20) where they are actually needed.

[0222] The robot system according to the present invention is not limited to the configuration and method of the embodiments described above; rather, all or part of each embodiment may be selectively combined to allow for various modifications to be made.

[0223] Meanwhile, the control method of a robot system according to an embodiment of the present invention can be implemented as processor-readable code on a processor-readable recording medium. A processor-readable recording medium includes all types of recording devices in which data that can be read by a processor is stored. Examples of processor-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and also include implementation in the form of a carrier wave, such as transmission via the Internet. Furthermore, the processor-readable recording medium may be distributed across networked computer systems, so that processor-readable code can be stored and executed in a distributed manner.

[0224] Furthermore, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above. Various modifications are possible by those skilled in the art without departing from the essence of the invention as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present invention. Explanation of the symbols

[0225] 10: Cloud Server 20: Edge Server 30: Robot

Claims

Claim 1 A cloud-based robot system comprising: a plurality of robots disposed in a plurality of arbitrarily divided spaces; a cloud server that generates and distributes a control base model applicable to the plurality of robots; and a plurality of edge servers each assigned to a space, transmitting and receiving with the cloud server, receiving the control base model, and upgrading it according to the plurality of robots in the space to control the plurality of robots with the upgraded control model, wherein the plurality of robots include robots of different types, and the edge servers receive the control base model, tune and execute the control base model according to the type of robot being controlled, perform deep learning error learning on the tuned control base model to generate an upgraded control model, and the upgraded control model is directly transmitted to other edge servers, and the cloud server obtains information regarding the upgraded control model, selects other edge servers to which the upgraded control model is applied, and enables the direct transmission of the upgraded control model between the edge servers. Claim 2 delete Claim 3 A cloud-based robot system according to claim 1, characterized in that the control base model has a respective control model for a plurality of functions of different types of robots packaged therein. Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 A cloud-based robot system according to claim 1, characterized in that the cloud server selects the edge server containing the robot to which the upgraded control model is applied and transmits the upgraded control model. Claim 8 A cloud-based robot system according to claim 7, characterized in that the edge server performs deep learning error learning to generate the upgraded control model when the error value exceeding a threshold is greater than or equal to a predetermined number while executing the control base model. Claim 9 A cloud-based robot system according to claim 1, characterized in that the cloud server classifies a plurality of edge servers into a plurality of groups and manages them, and one group is located within a predetermined distance or a predetermined response time. Claim 10 delete Claim 11 A cloud-based robot control method for controlling multiple robots placed in multiple arbitrarily divided spaces, comprising: generating a control base model applicable to the multiple robots in a cloud server; distributing the control base model to an edge server allocated to each space; upgrading the control base model according to the multiple robots in the space in the edge server; and directly transmitting the upgraded control model from the edge server to another edge server. A cloud-based robot control method comprising the steps of: controlling a plurality of robots with the upgraded control model at the edge server; wherein the plurality of robots include different types of robots; and the step of upgrading the control base model comprises: generating an upgraded control model by performing deep learning error learning on the tuned control base model; receiving the control base model at the edge server; and tuning the control base model according to the type of robot controlled by the edge server; obtaining information about the upgraded control model at the cloud server; selecting another edge server to which the upgraded control model is applied; transmitting information of the other edge server selected as the upgraded edge server; and further including the step of directly transmitting the upgraded control model from the upgraded edge server to the other edge server selected. Claim 12 delete Claim 13 A cloud-based robot control method according to claim 11, wherein the control base model generation step comprises generating and packaging respective control models for multiple functions of different types of robots. Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 A cloud-based robot control method according to claim 11, wherein the step of selecting the other edge server is characterized in that the cloud server selects an edge server containing a robot to which the upgraded control model is applied. Claim 18 A cloud-based robot control method according to claim 17, wherein the step of upgrading the control base model is characterized by performing deep learning error learning to generate the upgraded control model when the error value exceeding a threshold is greater than or equal to a predetermined number while running the control base model on the edge server. Claim 19 A cloud-based robot control method according to claim 11, further comprising the step of classifying and grouping a plurality of edge servers from the cloud server, wherein one group is located within a predetermined distance or a predetermined response time. Claim 20 delete

Images