Robot remote-control method, computing device, and computer program

Abstract

The present invention relates to a robot remote-control method performed in a computing device, and the method may comprise the steps of: distributing a service scenario for a service desired by a user to a robot; monitoring, in real time, the robot operating on the basis of the distributed service scenario; and remotely controlling an operation of the robot according to a monitoring result. Acknowledgement: This patent relates to research that was conducted with the support of the Korea Institute of Procurement and funded by the Government of the Republic of Korea (Public Procurement Service) in 2025. (No. RS-2025-16072908).

Description

Remote robot control method, computing device and computer program

[0001] The present invention relates to a remote robot control method, a computing device, and a computer program.

[0002] Recently, various types of robots are being used in real life and factory automation.

[0003] These robots have specialized control scenarios tailored to their specific purposes. In factory automation, motion control scenarios suited to repetitive tasks are primarily used, and these scenarios are designed to allow robots to perform repetitive tasks to maximize efficiency.

[0004] However, simple repetitive scenarios are not sufficient for service robots used in everyday life or in various places.

[0005] For example, guide robots must interact with users and provide information in real time, and delivery robots must avoid obstacles in their path and safely reach their destination.

[0006] For this reason, even for the same robot, the scenario for robot control often needs to be modified depending on the location where it is used or the service it provides, and sophisticated control is required even in complex environments.

[0007] However, existing remote robot control systems suffer from a lack of sufficient emergency response capabilities. This is because, when faced with an unexpected, dangerous situation, the robot's ability to independently recognize and respond to the situation is limited.

[0008] These issues are particularly significant for industrial robots and autonomous robots used in real-world applications. For example, factory automation robots lack the ability to immediately perform an emergency stop or detour when facing a collision risk due to mechanical errors or sensor malfunctions. Consequently, these limited response capabilities compromise the robot's safety and reliability, potentially posing a risk to workers and the surrounding environment.

[0009] Additionally, in existing remote control systems, if the network connection is lost or unstable, the robot may not function properly, stop, or perform unexpected actions, which may leave the robot in a dangerous state or cause it to stop working, seriously affecting the productivity and safety of the entire system.

[0010] To solve these problems, a remote robot control method with high response capabilities in various emergency situations is required.

[0011] The purpose of the present invention is to propose a remote robot control method, a computing device and a computer program.

[0012] Specifically, the present invention aims to propose a remote robot control method, computing device and computer program having high response capabilities in various emergency situations.

[0013] The purposes of the present invention are not limited to the purposes mentioned above, and other purposes and advantages of the present invention that are not mentioned can be understood by the following description and will be more clearly understood by the embodiments of the present invention.

[0014] In addition, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims.

[0015] A method for remotely controlling a robot performed in a computing device according to an embodiment of the present invention for achieving the above-described purpose may include a step of distributing a service scenario for a service desired by a user to a robot, a step of monitoring in real time a robot operating based on the distributed service scenario, and a step of remotely controlling the operation of the robot based on the monitoring result.

[0016] Additionally, the monitoring step may monitor at least one of the current situation of the robot, the current status, the performance status of a task within a service scenario, and the network status.

[0017] In addition, the step of remotely controlling the robot may include remotely controlling the robot to change its working mode according to the monitoring result, and the working mode may include an emergency stop mode, a multi-robot cooperation mode, a safety working mode, and an autonomous working mode.

[0018] In addition, the above remote control step can change the robot to a multi-robot cooperation mode in a situation where the work of the robot is delayed, and divide and assign the current work to robots adjacent to the robot.

[0019] Additionally, the remote control step can remotely control the robot to change to an autonomous operation mode when the network connection of the robot is unstable.

[0020] Additionally, the method may further include a step of providing a warning notification to the user when a specific event occurs based on the above monitoring results.

[0021] In addition, the method further includes a step of receiving a control command based on the monitoring result from the user, wherein the control command may be a control command for controlling the operation of the robot.

[0022] Meanwhile, the present invention implemented as a computer device includes a processor and a memory communicating with the processor, the memory storing commands that cause the processor to perform operations, and the operations may include an operation of distributing a service scenario for a service desired by a user to a robot, an operation of monitoring a robot operating based on the distributed service scenario in real time, and an operation of remotely controlling the operation of the robot based on the monitoring result.

[0023] Additionally, the monitoring operation may monitor at least one of the current situation of the robot, the current status, the performance status of a task within a service scenario, and the network status.

[0024] In addition, the operation of remotely controlling the robot may be performed to remotely change the working mode of the robot according to the monitoring result, and the working mode may include an emergency stop mode, a multi-robot cooperation mode, a safety working mode, and an autonomous working mode.

[0025] Additionally, the above remote control operation can change the robot to a multi-robot cooperation mode in a situation where the robot's work is delayed, and divide and assign the current work to robots adjacent to the robot.

[0026] Additionally, the above remote control operation can remotely control the robot to change to an autonomous operation mode when the network connection of the robot is unstable.

[0027] Additionally, the monitoring results may further include an action of providing a warning notification to the user when a specific event occurs.

[0028] In addition, the method further includes an operation of receiving a control command based on a monitoring result from the user, wherein the control command may be a control command for controlling the operation of the robot.

[0029] Meanwhile, a computer program stored in a computer-readable recording medium according to an embodiment of the present invention for achieving the above-described purpose may include a program code for performing the above-described remote robot control method.

[0030] In addition, a computer-readable recording medium according to an embodiment of the present invention for achieving the above-described purpose may record a computer program for executing the above-described remote robot control method.

[0031] According to the present invention, an optimal service scenario can be distributed in real time according to the status and environmental conditions of each robot, thereby enabling the robot to operate appropriately to the situation and increase work efficiency and accuracy.

[0032] In addition, the present invention can increase the efficiency and accuracy of the overall operation by monitoring and controlling the robot in real time while the robot performs the operation according to the service scenario.

[0033] In addition, the present invention provides various operation modes depending on the situation, thereby enabling safe and efficient operation even in unexpected situations.

[0034] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0035] Figure 1 is a schematic diagram showing the configuration of a robot control system according to one embodiment of the present invention.

[0036] Figure 2 is an exemplary diagram showing the configuration of a robot control system according to one embodiment of the present invention.

[0037] Figure 3 is a flowchart illustrating a scenario creation method according to one embodiment of the present invention.

[0038] Figure 4 is a flowchart illustrating a remote robot control method according to one embodiment of the present invention.

[0039] Figures 5 to 7 are exemplary diagrams showing the operation process of a robot according to one embodiment of the present invention.

[0040] Figure 8 is a flowchart illustrating a remote robot control method according to one embodiment of the present invention in more detail.

[0041] Figure 9 is an exemplary diagram showing a user interface according to one embodiment of the present invention.

[0042] Figure 10 is an exemplary diagram showing the configuration of a computing device according to one embodiment of the present invention.

[0043] The following merely illustrates the principles of the present invention. Therefore, those skilled in the art will be able to implement the principles of the present invention and invent various devices within the scope and spirit of the present invention, even if not explicitly described or illustrated herein. Furthermore, all conditional terms and embodiments listed herein are expressly intended, in principle, to facilitate understanding of the concepts of the present invention, and should be understood as being in no way limiting to the specifically enumerated embodiments and conditions.

[0044] The above-described objects, features and advantages will become more apparent through the following detailed description with reference to the attached drawings, so that a person having ordinary skill in the art to which the present invention pertains can easily practice the technical idea of ​​the present invention.

[0045] In addition, in describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

[0046] Hereinafter, various embodiments of the present invention will be described in detail with reference to the attached drawings.

[0047] Figure 1 is a schematic diagram showing the configuration of a robot control system according to one embodiment of the present invention.

[0048] Referring to FIG. 1, a robot control system (1000) according to the present invention may include a process for dynamically planning and optimizing the actions of a robot, and the robot control system (1000) may provide a robot control service aimed at automatically generating and coordinating the tasks of the robot through various steps, such as inputting scenarios / commands for robot control, action planning, and execution management. Here, a robot may refer to any type of robot that can automate specific tasks, extend human capabilities, and perform tasks in difficult-to-access locations.

[0049] For example, robots may include robots that perform exploration activities in difficult-to-access environments, robots that assist with precise surgery or treatment in the medical field, and robots that perform tasks such as assembling, packaging, welding, and painting products in manufacturing processes. In the present invention, the description will be based on robots that perform tasks such as cleaning, guiding, and delivery for the purpose of serving general consumers or providing services.

[0050] The robot control service of the robot control system (1000) can be broadly divided into three parts: a scenario process for inputting, outputting, generating, and managing scenarios for robot control; a work process for planning and assigning robot behavior trees (BTs) and tasks; and a simulation process for executing simulations. Each process can interact and be operated in an integrated manner to optimally achieve the user's desired results.

[0051] First, regarding the scenario process, a user using the robot control system (1000) can input scenarios or control commands for tasks to be performed by the robot through an app (APP, 11) that provides robot control services. The user-entered scenarios can be easily managed (edited, created, etc.) through an intuitive user interface and adjusted to various work environments and purposes. In this case, the user may refer to an employee (e.g., an on-site consulting employee) who contacts a customer to provide robot control services, or a customer who utilizes the robot control services.

[0052] The user's input scenario is created as a scenario for a service through several modification processes via the scenario module (1200) in the robot control system (1000), and the created scenario can be reflected in the robot's work plan and allocation through the work creation module (1400).

[0053] And, through the simulation module (1300), the service scenario can be simulated as a virtual robot in a virtual space (12) and then distributed as a robot that will provide the service desired by the user.

[0054]

[0055] Hereinafter, the overall operation of the robot control system (1000) will be described with reference to FIG. 2.

[0056] Figure 2 is an exemplary diagram showing the configuration of a robot control system according to one embodiment of the present invention.

[0057] Referring to FIG. 2, a user (21) can provide user input to a robot control system (1000) to create a scenario or manage (e.g., modify) an existing scenario through an interface module (1100) for robot control suitable for a desired service. Here, the interface module (1100) can interact with the user through a web browser or an app.

[0058] In addition, the scenario module (1200) can create a new scenario or modify an existing scenario based on user input entered through the interface module (1100), and can manage / modify various items related to the scenario (e.g., movement path, map, BGM, etc.).

[0059] Next, the service scenario generated in the scenario module (1200) and various data related to the service scenario can be provided to the task generation module (1400) via the data management module (1500). Here, the data management module (1500) is responsible for service management and data relay management, and can coordinate smooth data flow and tasks with other modules of the robot control system (1000).

[0060] In addition, the task creation module (1400) establishes task allocation and task plans for each robot using a service scenario and user input, creates a robot behavior tree (BT) and reflects it in the user's scenario to create a service scenario, and this service scenario can be provided to the simulation module (1300) via the data management module (1500).

[0061] Additionally, the simulation module (1300) performs services using a virtual robot in a virtual environment based on a service scenario, thereby verifying (or testing) the service scenario and acquiring various data (e.g., success or failure, spatial status, learning data, etc.). The various data acquired in this way are provided to the task generation module (1400) and can be used to optimize the service scenario.

[0062] In addition, the data management module (1500) can distribute service scenarios to a robot (22) via a relay module (1600). Here, the relay module (1600) is a module that supports safe and efficient data communication in a network environment and can perform packet relay and TLS mutual authentication.

[0063] Additionally, the data storage module (1700) can store various data generated or input during the operation of the robot control system (1000).

[0064] For example, the data storage module (1700) may store performance data, reference data, simulator data, environmental data, robot data, sensing data, service data, keywords matching main actions and sub-actions, etc.

[0065] Through each module of the robot control system (1000) described above, a user can create and modify various desired scenarios in real time, and by verifying and optimizing the proposed scenarios through simulation, a robot control service that can quickly respond to dynamic environmental changes can be provided.

[0066]

[0067] Next, a method for generating a scenario for robot control in a robot control system (1000) is described with reference to FIG. 3.

[0068] Figure 3 is a flowchart illustrating a scenario creation method according to one embodiment of the present invention.

[0069] Referring to FIG. 3, the robot control system (1000) can receive a draft scenario for a service to be performed by the robot from the user (S100). Here, the draft scenario serves as an initial input stage in the robot control system (1000) where the user defines the basic plan and requirements for the task to be performed by the robot. The user can set the robot's task goals and environmental conditions and outline the task to be performed by the robot. For example, the draft scenario may include information about the path, movements, etc. for the service to be performed by the robot.

[0070] Specifically, the interface module (1100) of the robot control system (100) supports various input methods such as a GUI (Graphical User Interface) input method, a voice input method, and a text input method so that a user can input a scenario intuitively and efficiently, and each input method will be described.

[0071] First, let's explain the GUI input method. Users can input scenarios through an intuitive graphical user interface. The GUI input method allows users to interact with the robot through drag-and-drop, button clicks, and other methods, allowing them to easily configure the robot's work path, target points, and task details.

[0072] For example, a user can specify a robot's movement path by dragging it on an on-screen map, and enter a draft scenario by selecting and adding the required tasks from a task list.

[0073] Next, the text input method is described. A user can input a scenario in text format. At this time, the robot control system (100) can recommend commands or settings for controlling the robot based on the text entered by the user, and the user can select the recommended commands or settings to input a draft scenario. This text input can be useful for accurately conveying detailed commands or settings.

[0074] Additionally, when explaining the voice input method, users can input draft scenarios using natural language. For example, a user can input a draft scenario for the robot by voice-referencing a command such as "Start cleaning in the living room at 10 o'clock and move to the kitchen after cleaning."

[0075] Meanwhile, the draft scenario input into the robot control system (1000) may be input by a consulting staff member who conducts consultation with a customer receiving the service.

[0076] Additionally, the robot control system (100) can receive a draft scenario from at least two input methods among a GUI (Graphical User Interface) input method, a voice input method, and a text input method.

[0077] In this case, the robot control system (100) can preprocess each draft scenario input through multiple input methods to generate a multi-modal draft scenario. Here, the multi-modal draft scenario may refer to a single comprehensive draft scenario that combines data collected through multiple input methods.

[0078] Specifically, the robot control system (100) can generate a multi-modal draft scenario in text format by organizing content inconsistencies or duplicate information between various input formats or resolving conflicts in draft scenarios input through various input methods.

[0079] For example, if a draft scenario such as “Deliver the document to the 3rd floor marketing department, then deliver coffee to the 5th floor conference room” is input by voice input, or “Designate the route from the 3rd floor marketing department to the 5th floor conference room by dragging it on the map within the building” is input by GUI input, the robot control system (100) can preprocess the input draft scenario to generate a multi-modal draft scenario such as “Move to the 3rd floor marketing department and place the document, then move to the 5th floor conference room and place the coffee there.”

[0080] By utilizing two or more input methods in this way, users can effectively input draft scenarios for their needs.

[0081] Additionally, the robot control system (1000) can detect keywords related to robot control from the input draft scenario (S200). Here, keywords are words or phrases with significant meaning in the user-entered commands or scenarios, and can be used to clearly define the robot's task goals and methods. For example, keywords can be words indicating the type, location, target, or action of the task to be performed by the robot, such as "movement," "arrival," "cleaning," "delivery," or "security."

[0082] Specifically, when a user inputs a draft scenario using voice and text input, the draft scenario is input in natural language, so the scenario module (1200) of the robot control system (1000) can utilize natural language processing (NLP) technology to interpret the syntax of the input sentence, identify verbs and nouns indicating major actions, and extract keywords related to robot control. At this time, the robot control system (1000) can analyze the context of the sentence, process polysemous and complex sentences, and appropriately detect keywords.

[0083] Furthermore, the robot control system (1000) can recognize and compensate for various input variations and differences in the user's pronunciation, intonation, and expression style through continuous learning, thereby accurately detecting keywords. For example, even if the user inputs the command "Start food delivery from the kitchen" differently depending on the expression style, such as "Bring food to the kitchen" or "Deliver to the kitchen," the robot control system (1000) can interpret these as having the same meaning and accurately detect keywords such as "delivery" and "kitchen."

[0084] That is, the robot control system (1000) can detect keywords by recognizing new input methods or pronunciation patterns through adaptive learning.

[0085] In addition, when a user inputs a draft scenario using a GUI input method, when configuring the scenario, he or she is provided with options for settings, commands, etc. for the robot's operations, and each option is matched with a predefined keyword, and the robot control system (1000) can detect keywords related to robot control in the draft scenario through the matched keywords.

[0086] Additionally, when a user inputs a draft scenario in multiple input modes, the robot control system (1000) can detect keywords related to robot control from the multi-modal draft scenario.

[0087] Additionally, the robot control system (1000) can recommend the robot's main action and sub-action to the user based on the detected keywords (S300). Here, the robot's main action represents the main task performed by the robot, and the sub-action represents a detailed task that supports or assists the main action.

[0088] Specifically, a main action can consist of actions that represent the primary task or goal the robot must perform. For example, a main action can be comprised of essential parts of a task, such as "Move," "Arrive," "Start," "Pick up a tray," or "Return to a waiting area," and can define the goals the robot must achieve.

[0089] Additionally, sub-actions are subtasks that assist the main action and can be composed of detailed tasks to support the execution of the main action. These can be defined as necessary components to increase the efficiency and success rate of the main action, such as "BGM operation," "LED operation," "TTS playback," "tray check," and "battery check."

[0090] That is, the main action represents the main task or goal that the robot must perform, and the sub-action can mean a detailed sub-task to assist the main action and support its execution.

[0091] Additionally, the robot control system (1000) can recommend to the user the main action and sub-action of the robot that are matched by each detected keyword.

[0092] In addition, the robot control system (1000) can select the main action and sub-action of the robot and recommend them to the user by considering not only the detected keywords but also various data (e.g., robot data, environmental data, etc.).

[0093] Next, the robot control system (1000) can verify user approval for the recommended main and sub-actions (S400). Specifically, the robot control system (1000) considers approval complete when the user presses the "Final Confirmation" button. However, if the user selects only a portion of the actions or modifies internal parameters, the robot control system (1000) may consider approval disapproved.

[0094] If the user does not approve (S400, NO), the robot control system (1000) may receive the user's modified scenario for the service (S600). Here, the modified scenario refers to the robot's work plan updated based on the draft scenario to accommodate changes in user needs or environmental conditions. The modified scenario may also refer to a step that optimizes or refines the robot's action plan by reflecting additional user input or feedback based on the work goals and conditions defined in the draft scenario.

[0095] These revised scenarios can be entered using the same input method as the draft scenario entry step (S100) described above. For example, revised scenarios can be entered that include changing the order of recommended main actions and sub-actions, modifying internal parameters of sub-actions, or selecting other main actions or sub-actions.

[0096] In addition, the robot control system (1000) can re-detect keywords in the input modification scenario (S200) and re-recommend main actions and sub-actions (S300) based on the re-detected keywords. At this time, the keyword re-detection step (S200) may be omitted depending on the input modification scenario.

[0097] That is, the above-described keyword detection step (S200) and recommendation step (S300) can be repeatedly performed according to user approval.

[0098] Meanwhile, if the user approves (S400, NO), the robot control system (1000) can generate a service scenario based on the determined main action and sub-action (S500).

[0099] Specifically, the task generation module (1400) of the robot control system (1000) can generate a service scenario by integrating the robot's action tree, task plan, and assignment based on the determined main and sub-actions of the scenario. Here, the service scenario is a final robot task plan adjusted to the user's needs and environmental conditions, and means a comprehensive plan for the robot's task performance by integrating the robot's action tree, task plan, and assignment.

[0100] These service scenarios can be deployed to each robot, and each robot can operate based on the service scenario.

[0101] Additionally, the robot control system (1000) can remotely control the robot as needed, and this is further described with reference to FIG. 4.

[0102] Figure 4 is a flowchart illustrating a remote robot control method according to one embodiment of the present invention.

[0103] Referring to FIG. 4, the robot control system (1000) can distribute a service scenario to each robot (S1000).

[0104] Specifically, the data management module (1500) of the robot control system (1000) can distribute the service scenario generated in the task generation module (1400) to each robot through the relay module (1600).

[0105] Additionally, the robot control system (1000) can analyze the status, location, network status, and work situation of each robot in real time to deploy an optimal service scenario for each robot. For example, even in the same work environment, the robot control system (1000) can adjust and deploy a service scenario in real time based on factors such as the battery status of each robot or the density of surrounding obstacles.

[0106] Meanwhile, robots that have received service scenarios from the robot control system (1000) can perform services according to the service scenarios. For example, a robot that has received a service scenario for a cleaning service can move along a cleaning area and path preset in the service scenario and begin and complete cleaning tasks at a set time.

[0107] Next, the robot control system (1000) can monitor in real time the robot operating based on the deployed service scenario (S2000).

[0108] Specifically, the robot control system (1000) can monitor various robot items in real time by receiving various data measured in real time from each robot through wireless communication with the robot via a relay module (1600). Here, the monitoring items may include the robot's current status, current state, task execution status within a service scenario, network status, and occurrence of specific events, and each item is described below.

[0109] First, among the monitoring items, the current situation of the robot refers to the external environment or conditions the robot faces. This can include external factors with which the robot must interact. For example, the current situation of the robot may include situations in which it encounters obstacles (e.g., obstacles not included in the service scenario or obstacles in other locations), climate change, or crowded areas.

[0110] Additionally, among the monitoring items, the current state of the robot refers to the internal or operational status of the robot itself, indicating how its hardware and software systems are currently operating. For example, the current state of the robot may include the robot's battery status, sensor status, and temperature status.

[0111] Additionally, among the monitoring items, the performance status of a task within a service scenario refers to the progress of a specific task within the service scenario being performed by the robot. This can mean monitoring the extent to which the robot has completed a given task according to the service scenario. For example, the performance status of a task within a service scenario may include a task completion status, a task interruption status, or a task progress status.

[0112] Additionally, among the monitoring items, the network status refers to the status of the robot's connection to the robot control system (1000), and monitors the status of the robot exchanging data over the network. For example, the network status may include network connection delay, connection loss, and normal connection.

[0113] That is, the robot control system (1000) can recognize and judge various situations by monitoring at least one of the current situation of the robot, the current status, the performance status of a task within a service scenario, and the network status.

[0114] Next, the robot control system (1000) can remotely control the robot's movements based on the monitoring results (S3000).

[0115] Specifically, the data management module (1500) of the robot control system (1000) can remotely control the robot to change its operating mode based on the monitoring results. Here, the operating mode may include a service scenario mode, an emergency stop mode, a multi-robot collaboration mode, a safety operation mode, an autonomous operation mode, and a remote control mode, each of which is described below.

[0116] First, the service scenario mode is a mode in which the robot operates according to the service scenario, operates based on a set path, work sequence, and work goal, and performs pre-planned operations under expected environmental conditions. It may be the basic operating mode of a robot that has been distributed with a service scenario.

[0117] Additionally, the emergency stop mode is a mode that immediately stops all operations when the robot detects an unexpected emergency or risk factor, and can be applied in situations such as collisions with obstacles or sensor errors.

[0118] In addition, multi-robot cooperation mode is a mode in which multiple robots cooperate with each other to achieve the same task goal, and refers to a mode in which multiple robots divide up complex tasks and perform them, or cooperate to perform a single task simultaneously.

[0119] Additionally, the safe work mode is a mode that allows the robot to perform tasks as safely as possible, and is a mode that limits the robot's speed and movement to reduce the risk of accidents.

[0120] Additionally, autonomous operation mode is a mode in which the robot perceives the environment, makes judgments, and performs tasks on its own without direct external intervention, and AI and autonomous driving technology can be applied.

[0121] Additionally, the remote control mode is a mode in which the robot is controlled remotely according to the user's real-time commands, and means a mode in which the robot's movements or operations are performed in real time according to the user's input.

[0122] Next, examples of each operating mode will be described with additional reference to FIGS. 5 to 7.

[0123] Figures 5 to 7 are exemplary diagrams showing the operation process of a robot according to one embodiment of the present invention.

[0124] First, referring to FIG. 5, a robot (51) that has received a service scenario from a robot control system (1000) operates in a service scenario mode, and the robot (51) can provide a service by moving to the first table (52) and the second table (53) while avoiding a movable obstacle (e.g., a person) or a fixed obstacle according to the service scenario.

[0125] If, as in FIG. 6, a movable obstacle (62) suddenly appears on the existing path according to the service scenario and it is monitored that the service scenario mode cannot be operated, the robot control system (1000) remotely controls the operation mode of the robot (61) to be changed to an autonomous operation mode, and the robot (61) recognizes obstacles around the robot (61) according to the autonomous operation mode, creates a new movement path on its own, and moves according to the new movement path to provide service to each table.

[0126] In addition, when the operation of the first robot (71) is monitored as being delayed as in FIG. 7, the robot control system (1000) can remotely control the operation mode of the first robot (71) to be changed to a multi-robot cooperation mode.

[0127] In addition, the robot control system (1000) can divide and assign the work that is not in progress among the work in the service scenario of the first robot (71) that is delayed to the adjacent second robot (72), and the first robot (71) can provide the service to the first table (73), and the second robot (72) can provide the service to the second table (74). At this time, the robot control system (1000) can divide the work in the service scenario so that the finishing times of the work of the first robot (71) and the second robot (72) are minimized.

[0128] That is, when the robot's work is delayed, the robot control system (1000) can change the operating mode of the robot whose work is delayed to a multi-robot cooperation mode and divide and assign the current work to other robots adjacent to the robot whose work is delayed.

[0129] Additionally, the robot control system (1000) can be remotely controlled to enter emergency stop mode, safety operation mode, and remote control mode based on monitoring results or needs, as the above-described examples are merely examples. For example, the robot control system (1000) can remotely control the robot to change to autonomous operation mode when the robot's network connection is unstable.

[0130] That is, the robot control system (1000) can remotely control the robot by automatically changing the operating mode of the robot according to the monitoring results.

[0131] Additionally, the robot control system (1000) may provide a warning notification to the user via the interface module (1100) when a specific event occurs based on the monitoring results. Here, the specific event may include an overheating event, a collision detection event, a task failure (or success) event, a maintenance requirement event, a data transmission failure event, etc.

[0132] For example, if the monitoring results indicate that a physical collision has been detected between the robot and the robot control system (1000), the robot control system (1000) may determine that a collision detection event has occurred and provide a notification to the user (or administrator) to stop the robot's movement or move backward.

[0133] Meanwhile, the robot control system (1000) provides real-time monitoring results to the user and can also remotely control the robot by receiving control commands from the user. This will be further described with reference to FIG. 8.

[0134] Figure 8 is a flowchart illustrating a remote robot control method according to one embodiment of the present invention in more detail.

[0135] Referring to FIG. 8, the robot control system (1000) can provide real-time monitoring results (S2100).

[0136] Specifically, the data management module (1500) of the robot control system (1000) can provide real-time monitoring results to the user through the interface module (1100). At this time, the robot control system (1000) can provide real-time monitoring results through the user interface, and this will be further described with reference to FIG. 9.

[0137] Figure 9 is an exemplary diagram showing a user interface according to one embodiment of the present invention.

[0138] Referring to FIG. 9, the robot control system (1000) can provide real-time monitoring results of the robot through a user interface.

[0139] Specifically, the user interface may display real-time camera footage of the robot, sensor data such as thermal imaging data, a map of the surroundings, and a route plan.

[0140] Additionally, the user interface may include various data such as the robot's work progress status, network status, etc.

[0141] Next, the robot control system (1000) can receive a control command from the user to control the operation of the robot (S2200).

[0142] Specifically, the robot control system (1000) can receive control commands for controlling the operation of the robot from a user in real time through an interface module (1100).

[0143] For example, a user interface that provides real-time monitoring results, such as in Fig. 9, may be provided with an input area for controlling the robot's movements, and the user may input control commands for controlling various movements of the robot in real time through the input area.

[0144] Next, the robot control system (1000) can correct the control command input from the user (S2300).

[0145] Specifically, the data management module (1500) of the robot control system (1000) can correct the control command input by the user by taking into account the transmission delay of the real-time camera image. Here, the real-time camera image may be viewed by the user with a slight difference from the actual robot situation due to reasons such as network conditions or transmission delays. Although the user inputs the control command based on the real-time camera image, if a transmission delay occurs, a time difference may occur between the actual robot situation and the state displayed on the screen.

[0146] For example, even if a robot is commanded to stop in front of an obstacle, if the video the user sees is already delayed, there is a high probability that the robot has already reached that point.

[0147] Accordingly, the robot control system (1000) can compare and analyze the time at which a user command is input and the actual state of the robot to compensate for this delay, and apply a pre-calculated compensation value at the time at which the command is executed.

[0148] Additionally, the robot control system (1000) can correct the user's control command by taking into account the time the control command is transmitted to the robot.

[0149] That is, the robot control system (1000) can correct the control commands input by the user to match the user's intention. This minimizes errors caused by transmission delays and enables more accurate remote control.

[0150] In addition, the robot control system (1000) may not simply calculate the time difference in the process of compensating for the transmission delay that occurs between the user's command and the robot's response, but may also apply a robot motion prediction algorithm. At this time, the robot control system (1000) may consider additional variables that may occur while the robot is executing the command (e.g., unexpected obstacles, sudden changes in battery status, etc.) to more precisely adjust the execution timing of the user's command. For example, the robot control system (1000) estimates the location of the robot in advance before the user's control command is transmitted to the robot based on predicted movement data from the robot's driving path, and generates a correction value reflecting this.

[0151] By doing this, even if the user issues a stop command close to an obstacle, the robot control system can control the robot more precisely to avoid actually colliding with the obstacle.

[0152] Additionally, the robot control system (1000) can automatically adjust the priority of control commands by monitoring in real time not only the time at which control commands are transmitted to the robot, but also network delays and communication instability. For example, a stop command in an emergency situation can be given a higher priority than other commands, allowing for immediate processing with minimal transmission delay.

[0153] This prevents urgent commands (e.g., emergency stop commands) from being executed late due to network delays, enabling more immediate responses.

[0154] Additionally, the robot control system (1000) utilizes an artificial intelligence-based correction algorithm to analyze control commands entered by the user in real time, enabling the correction process to be performed more precisely based on this analysis. The algorithm continuously learns past control commands and the resulting data, enabling it to analyze the pattern of commands issued by the user in specific situations and automatically adjust correction values ​​accordingly.

[0155] For example, each time a user manipulates a robot, the control commands input and the robot's actual responses can be recorded, and this data can be analyzed to determine whether the commands were executed as intended or whether unexpected results occurred. This process utilizes a reinforcement learning algorithm, allowing the robot control system (1000) to gradually learn optimal correction methods based on feedback for each user command.

[0156] That is, through a reinforcement learning algorithm, the robot control system (1000) can continuously learn the user's control commands and results, and automatically adjust the optimal correction value in real time.

[0157] Meanwhile, the robot control system (1000) of the present invention or each module within the robot control system (1000) can be implemented as a computing device, and this will be described with reference to FIG. 10.

[0158] Figure 10 is an exemplary diagram showing the configuration of a computing device according to one embodiment of the present invention.

[0159] Referring to FIG. 10, in some embodiments of the present invention, the robot control system (1000) may be implemented in the form of a computing device.

[0160] At least one of each module constituting the robot control system (1000) is implemented on a general-purpose computing processor and thus may include a processor (1008), an input / output I / O (1002), a memory (1004), an interface (1006), and a bus (1014). The processor (1008), the input / output device (1002), the memory (1004), and / or the interface (1006) may be coupled to each other via the bus (1014). The bus (1014) corresponds to a path through which data is transferred.

[0161] Specifically, the processor (1008) may include at least one of a Central Processing Unit (CPU), a Micro Processor Unit (MPU), a Micro Controller Unit (MCU), a Graphic Processing Unit (GPU), a microprocessor, a digital signal processor, a microcontroller, an application processor (AP), and logic elements capable of performing functions similar thereto.

[0162] The input / output device (1002) may include at least one of a keypad, a keyboard, a touchscreen, and a display device. The memory (1004) may store data and / or programs.

[0163] The interface (1006) may perform a function of transmitting data to or receiving data from a communication network. The interface (1006) may be wired or wireless. For example, the interface (1006) may include an antenna or a wired / wireless transceiver. The memory (1004) may further include high-speed DRAM and / or SRAM, etc., as a volatile operating memory that enhances the operation of the processor (1008) while protecting personal information.

[0164] Additionally, the memory (1004) stores programming and data configurations that provide the functionality of some or all of the modules described herein. For example, it may include logic for performing selected aspects of the remote robot control method according to the present embodiment.

[0165] A program or application is loaded with a set of instructions including each operation of the present invention stored in the memory (1004) and the processor is enabled to perform each operation. Here, the operation may include an operation of distributing a service scenario for a service desired by a user to a robot, an operation of monitoring a robot operating based on the distributed service scenario in real time, an operation of remotely controlling the operation of the robot based on the monitoring result, an operation of providing a warning notification to the user when a specific event occurs based on the monitoring result, an operation of receiving a control command based on the monitoring result from the user, an operation of providing the real-time monitoring result to the user, an operation of correcting the user's control command, etc.

[0166] The various embodiments described herein may be implemented in a recording medium readable by a computer or similar device, for example, using software, hardware, or a combination thereof.

[0167] In terms of hardware implementation, the embodiments described herein can be implemented using at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors, controllers, micro-controllers, microprocessors, and other electrical units for performing functions. In some cases, the embodiments described herein can be implemented as a control module itself.

[0168] In a software implementation, the procedures and functions described herein, as well as other embodiments, may be implemented as separate software modules. Each of these software modules may perform one or more of the functions and operations described herein. The software code may be implemented as a software application written in a suitable programming language. The software code may be stored in a memory module and executed by a control module.

[0169] The above description is merely an example of the technical idea of ​​the present invention, and those skilled in the art will appreciate that various modifications, changes, and substitutions can be made without departing from the essential characteristics of the present invention.

[0170] Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate, rather than limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by these embodiments and the accompanying drawings. The protection scope of the present invention should be interpreted by the following claims, and all technical concepts within the scope equivalent thereto should be interpreted as being included within the scope of the rights of the present invention.

Claims

1. A method for remotely controlling a robot performed on a computing device, A step of deploying a service scenario for a service desired by the user to a robot; A step of real-time monitoring of a robot operating based on a deployed service scenario; A remote robot control method comprising a step of remotely controlling the operation of the robot according to the monitoring results.

2. In paragraph 1, A remote robot control method characterized in that the monitoring step monitors at least one of the current situation of the robot, the current status, the performance status of a task within a service scenario, and the network status.

3. In paragraph 1, The step of remotely controlling the robot includes remotely controlling the robot to change its working mode according to the monitoring result. A remote robot control method, characterized in that the above working modes include an emergency stop mode, a multi-robot cooperation mode, a safety working mode, and an autonomous working mode.

4. In paragraph 3, A remote robot control method characterized in that the above remote control step changes the robot to a multi-robot cooperation mode when the work of the robot is delayed, and divides and assigns the current work to robots adjacent to the robot.

5. In paragraph 1, A remote robot control method characterized in that the above remote controlling step remotely controls the robot to change to an autonomous operation mode when the network connection of the robot is unstable.

6. In paragraph 1, A remote robot control method, further comprising a step of providing a warning notification to a user when a specific event occurs based on the above monitoring results.

7. In paragraph 1, A remote robot control method, further comprising: a step of receiving a control command based on the monitoring result from the user; wherein the control command is a control command for controlling the operation of the robot.

8. Processor, and including a memory communicating with the processor, The above memory stores instructions that cause the processor to perform operations, The above actions are actions of distributing a service scenario for a service desired by a user to a robot; Actions to monitor robots in real time based on deployed service scenarios; A computing device comprising an operation for remotely controlling the operation of the robot according to the monitoring results.

9. In paragraph 8, A computing device characterized in that the above monitoring operation monitors at least one of the current situation of the robot, the current status, the performance status of a task within a service scenario, and the network status.

10. In paragraph 8, The operation of remotely controlling the robot is to remotely control the robot to change its working mode according to the monitoring result, A computing device characterized in that the above working modes include an emergency stop mode, a multi-robot collaboration mode, a safety working mode, and an autonomous working mode.

11. In paragraph 10, A computing device characterized in that the above remote control operation changes the robot to a multi-robot cooperation mode when the work of the robot is delayed, and divides and assigns the current work to robots adjacent to the robot.

12. In paragraph 8, A computing device characterized in that the above remote control operation remotely controls the robot to change to an autonomous operation mode when the network connection of the robot is unstable.

13. In paragraph 8, A computing device further comprising: an operation for providing a warning notification to a user when a specific event occurs based on the above monitoring results.

14. In paragraph 8, A computing device further comprising a step of receiving a control command based on the monitoring result from the user, wherein the control command is a control command for controlling the operation of the robot.

15. A computer-readable recording medium storing a program for performing a remote robot control method according to any one of paragraphs 1 to 7.

16. A program stored on a computer-readable recording medium including a program code for executing a remote robot control method according to any one of paragraphs 1 to 7.

Images