Method, computing device, and computer program for generating scenario for robot control

Abstract

The present invention relates to a method, performed by a computing device, for generating a scenario for robot control, wherein the method may comprise the steps of: receiving, from a user, a draft scenario for a service to be performed by a robot; detecting keywords related to robot control from the received draft scenario; recommending a main action and a sub-action of the robot to the user on the basis of the detected keywords; and generating a scenario for the service in response to user approval of the recommendation. Acknowledgement: This patent was supported by the Korea Public Procurement Research Institute with funding from the Government (Public Procurement Service) in 2025. (No.RS-2025-16072908).

Description

Method for generating scenarios for robot control, computing device and computer program

[0001] The present invention relates to a method for generating a scenario for robot control, a computing device, and a computer program.

[0002] Recently, various types of robots are being used in real-world applications and factory automation. Each robot has specialized control scenarios tailored to its specific purpose. In factory automation, motion control scenarios suited to repetitive tasks are primarily used. These scenarios are designed to allow robots to perform repetitive tasks to maximize efficiency.

[0003] However, simple repetitive scenarios are not enough for service robots used in daily life or in various places.

[0004] For example, guide robots, delivery robots, and entertainment robots require more complex and flexible control scenarios because they require various actions and responses depending on the environment and situation.

[0005] Additionally, even for the same robot, there are many cases where the scenario for robot control needs to be modified depending on the location where it is used or the service provided.

[0006] Existing methods for generating scenarios to control robots were mainly designed manually by experts or automated using advanced algorithms.

[0007] However, the manual design method requires the deep knowledge and experience of experts, making it difficult for general users to access, and creating scenarios manually takes a lot of time and is highly prone to errors.

[0008] Furthermore, automated scenario generation methods increase time efficiency by using complex algorithms to generate scenarios. However, they require expensive hardware and software and are only usable by users with a high level of technical understanding. This results in a high cost burden and limited accessibility for general users.

[0009] Moreover, robots possess diverse inputs and outputs, making them often difficult to understand and handle using existing methods. Especially for service robots, a scenario generation method capable of reflecting diverse behaviors in real time in response to environmental changes is essential.

[0010] However, current scenario generation methods for robot control do not sufficiently meet these requirements and show limitations in flexibility and adaptability.

[0011] To address these challenges, we need an intuitive and user-friendly scenario creation method that customers can easily use.

[0012] The purpose of the present invention is to propose a method for generating a scenario for robot control, a computing device, and a computer program.

[0013] The objectives of the present invention are not limited to those mentioned above. Other objectives and advantages of the present invention not mentioned above can be understood through the following description and will be more clearly understood through the embodiments of the present invention. Furthermore, it will be readily apparent that the objectives and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims.

[0014] In order to achieve the above-described purpose, a method for generating a scenario for robot control performed in a computing device according to an embodiment of the present invention may include a step of receiving a draft scenario for a service to be performed by a robot from a user, a step of detecting keywords related to the robot control from the input draft scenario, a step of recommending a main action and a sub-action of the robot to the user based on the detected keywords, and a step of generating a scenario for the service based on user approval of the recommendation.

[0015] In addition, the method further includes a step of receiving a modification scenario for the service based on user approval for the recommendation, and the step of detecting the keyword can re-detect the keyword in the modification scenario.

[0016] Additionally, the step of recommending to the user may be repeatedly performed according to the user's approval.

[0017] In addition, the above-mentioned recommended step can analyze the context of the keyword and initially select main actions and sub-actions related to the control of the robot corresponding to the keyword.

[0018] In addition, the above-mentioned recommended step can secondarily select main actions and sub-actions related to the control of the robot that has been first selected based on environmental data about the place where the service will be provided and robot data about the function of the robot.

[0019] Additionally, the above recommended step can tertiarily select main actions and sub-actions related to the control of the second-selected robot based on user-specific selection data.

[0020] Additionally, the user-specific selection data may include the frequency of selection of main actions and sub-actions selected by other users.

[0021] Meanwhile, the present invention includes a processor and a memory communicating with the processor, wherein the memory stores commands that cause the processor to perform operations, and the operations may include an operation of receiving a draft scenario for a service to be performed by a robot from a user, an operation of detecting keywords related to robot control from the input draft scenario, an operation of recommending a main action and a sub-action of the robot to the user based on the detected keywords, and an operation of generating a scenario for the service based on user approval of the recommendation.

[0022] In addition, the method further includes an action of receiving a modification scenario for the service based on user approval for the recommendation, and the action of detecting the keyword can re-detect the keyword in the modification scenario.

[0023] In addition, the above recommended action can analyze the context of the keyword and initially select main actions and sub-actions related to the control of the robot corresponding to the keyword.

[0024] In addition, the above recommended actions can secondarily select main actions and sub-actions related to the control of the robot that have been first selected based on environmental data about the place where the service will be provided and robot data about the function of the robot.

[0025] Additionally, the above recommended actions can be tertiarily selected based on user-specific selection data, including main actions and sub-actions related to the control of the secondary-selected robot.

[0026] Additionally, the user-specific selection data may include the frequency of selection of main actions and sub-actions selected by other users.

[0027] According to the present invention, the scenario creation process can be drastically shortened by recommending main actions and sub-actions of a scenario for robot control based on keywords in a scenario input by a user.

[0028] Additionally, users can input the overall scenario of the service that the robot will provide using various input methods.

[0029] In addition, the present invention provides a graphical user interface that allows a user to directly design a scenario using drag and drop, thereby providing an environment that is easily accessible even to non-experts.

[0030] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0031] Figure 1 is a schematic diagram showing the configuration of a robot control system according to one embodiment of the present invention.

[0032] Figure 2 is an exemplary diagram showing the configuration of a robot control system according to one embodiment of the present invention.

[0033] Figure 3 is a flowchart illustrating a scenario creation method according to one embodiment of the present invention.

[0034] Figure 4 is an exemplary diagram showing a scenario input method according to one embodiment of the present invention.

[0035] Figures 5 and 6 are flowcharts specifically illustrating a scenario generation method according to one embodiment of the present invention.

[0036] FIG. 7 is an exemplary diagram showing a graphical user interface according to one embodiment of the present invention.

[0037] Figure 8 is an exemplary diagram showing the configuration of a computing device according to one embodiment of the present invention.

[0038] The following merely illustrates the principles of the present invention. Therefore, those skilled in the art will be able to implement the principles of the present invention and invent various devices within the scope and spirit of the present invention, even if not explicitly described or illustrated herein. Furthermore, all conditional terms and embodiments listed herein are expressly intended, in principle, to facilitate understanding of the concepts of the present invention, and should be understood as being in no way limiting to the specifically enumerated embodiments and conditions.

[0039] The above-described objects, features and advantages will become more apparent through the following detailed description with reference to the attached drawings, so that a person having ordinary skill in the art to which the present invention pertains can easily practice the technical idea of ​​the present invention.

[0040] In addition, in describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

[0041] Hereinafter, various embodiments of the present invention will be described in detail with reference to the attached drawings.

[0042] Figure 1 is a schematic diagram showing the configuration of a robot control system according to one embodiment of the present invention.

[0043] Referring to FIG. 1, a robot control system (1000) according to the present invention may include a process for dynamically planning and optimizing the actions of a robot, and the robot control system (1000) may provide a robot control service aimed at automatically generating and coordinating the tasks of the robot through various steps, such as inputting scenarios / commands for robot control, action planning, and execution management. Here, a robot may refer to any type of robot that can automate specific tasks, extend human capabilities, and perform tasks in difficult-to-access locations.

[0044] For example, robots may include robots that perform exploration activities in difficult-to-access environments, robots that assist with precise surgery or treatment in the medical field, and robots that perform tasks such as assembling, packaging, welding, and painting products in manufacturing processes. In the present invention, the description will be based on robots that perform tasks such as cleaning, guiding, and delivery for the purpose of serving general consumers or providing services.

[0045] The robot control service of the robot control system (1000) can be broadly divided into three parts: a scenario process for inputting, outputting, generating, and managing scenarios for robot control; a work process for planning and assigning robot behavior trees (BTs) and tasks; and a simulation process for executing simulations. Each process can interact and be operated in an integrated manner to optimally achieve the user's desired results.

[0046] First, regarding the scenario process, a user using the robot control system (1000) can input scenarios or control commands for tasks to be performed by the robot through an app (APP, 11) that provides robot control services. The user-entered scenarios can be easily managed (edited, created, etc.) through an intuitive user interface and adjusted to various work environments and purposes. In this case, the user may refer to an employee (e.g., an on-site consulting employee) who contacts a customer to provide robot control services, or a customer who utilizes the robot control services.

[0047] The user's input scenario is created as a scenario for a service through several modification processes via the scenario module (1200) in the robot control system (1000), and the created scenario can be reflected in the robot's work plan and allocation through the work creation module (1400).

[0048] And, through the simulation module (1300), the service scenario can be simulated as a virtual robot in a virtual space (12) and then distributed as a robot that will provide the service desired by the user.

[0049]

[0050] Hereinafter, the overall operation of the robot control system (1000) will be described with reference to FIG. 2.

[0051] Figure 2 is an exemplary diagram showing the configuration of a robot control system according to one embodiment of the present invention.

[0052] Referring to FIG. 2, a user (21) can provide user input to a robot control system (1000) to create a scenario or manage (e.g., modify) an existing scenario through an interface module (1100) for robot control suitable for a desired service. Here, the interface module (1100) can interact with the user through a web browser or an app.

[0053] In addition, the scenario module (1200) can create a new scenario or modify an existing scenario based on user input entered through the interface module (1100), and can manage / modify various items related to the scenario (e.g., movement path, map, BGM, etc.).

[0054] Next, the service scenario generated in the scenario module (1200) and various data related to the service scenario can be provided to the task generation module (1400) via the data management module (1500). Here, the data management module (1500) is responsible for service management and data relay management, and can coordinate smooth data flow and tasks with other modules of the robot control system (1000).

[0055] In addition, the task creation module (1400) establishes task allocation and task plans for each robot using a service scenario and user input, creates a robot behavior tree (BT) and reflects it in the user's scenario to create a service scenario, and this service scenario can be provided to the simulation module (1300) via the data management module (1500).

[0056] Additionally, the simulation module (1300) performs services using a virtual robot in a virtual environment based on a service scenario, thereby verifying (or testing) the service scenario and acquiring various data (e.g., success or failure, spatial status, learning data, etc.). The various data acquired in this way are provided to the task generation module (1400) and can be used to optimize the service scenario.

[0057] In addition, the data management module (1500) can distribute service scenarios to a robot (22) via a relay module (1600). Here, the relay module (1600) is a module that supports safe and efficient data communication in a network environment and can perform packet relay and TLS mutual authentication.

[0058] Additionally, the data storage module (1700) can store various data generated or input during the operation of the robot control system (1000).

[0059] For example, the data storage module (1700) may store performance data, reference data, simulator data, environmental data, robot data, sensing data, service data, keywords matching main actions and sub-actions, etc.

[0060] Through each module of the robot control system (1000) described above, a user can create and modify various desired scenarios in real time, and by verifying and optimizing the proposed scenarios through simulation, a robot control service that can quickly respond to dynamic environmental changes can be provided.

[0061] Next, a method for generating a scenario for robot control in a robot control system (1000) is described with reference to FIG. 3.

[0062] Figure 3 is a flowchart illustrating a scenario creation method according to one embodiment of the present invention.

[0063] Referring to FIG. 3, the robot control system (1000) can receive a draft scenario for a service to be performed by the robot from the user (S100). Here, the draft scenario is an initial input stage in the robot control system (1000) where the user defines the basic plan and requirements for the task to be performed by the robot. The user can set the robot's task goals and environmental conditions and provide an outline of the task to be performed by the robot. For example, the draft scenario may include information about the path, movements, etc. for the service to be performed by the robot.

[0064] Specifically, the interface module (1100) of the robot control system (100) supports various input methods such as a GUI (Graphical User Interface) input method, a voice input method, and a text input method so that a user can input a scenario intuitively and efficiently, and each input method will be described.

[0065] First, let's explain the GUI input method. Users can input scenarios through an intuitive graphical user interface. The GUI input method allows users to interact with the robot through drag-and-drop, button clicks, and other methods, allowing them to easily configure the robot's work path, target points, and task details.

[0066] For example, a user can specify a robot's movement path by dragging it on an on-screen map, and enter a draft scenario by selecting and adding the required tasks from a task list.

[0067] Next, the text input method is described. A user can input a scenario in text format. At this time, the robot control system (100) can recommend commands or settings for controlling the robot based on the text entered by the user, and the user can select the recommended commands or settings to input a draft scenario. This text input can be useful for accurately conveying detailed commands or settings.

[0068] Additionally, when explaining the voice input method, users can input draft scenarios using natural language. For example, a user can input a draft scenario for the robot by voice-referencing a command such as "Start cleaning in the living room at 10 o'clock and move to the kitchen after cleaning."

[0069] Meanwhile, the draft scenario input into the robot control system (1000) may also be input by a consulting staff member conducting a consultation with a customer receiving the service. For example, as shown in FIG. 4, a customer may convey a draft scenario to an on-site consulting staff member, stating, "After food delivery, I want to check for spilled food in a drawer at a specific location." The on-site consulting staff member may then input the draft scenario into the robot control system (1000) using a GUI input method, voice input method, or text input method, based on the customer's intent, through a conversation with the customer.

[0070] Additionally, the robot control system (100) can receive a draft scenario from at least two input methods among a GUI (Graphical User Interface) input method, a voice input method, and a text input method.

[0071] In this case, the robot control system (100) can preprocess each draft scenario input through multiple input methods to generate a multi-modal draft scenario. Here, the multi-modal draft scenario may refer to a single comprehensive draft scenario that combines data collected through multiple input methods.

[0072] Specifically, the robot control system (100) can generate a multi-modal draft scenario in text format by organizing content inconsistencies or duplicate information between various input formats or resolving conflicts in draft scenarios input through various input methods.

[0073] For example, if a draft scenario such as “Deliver the document to the 3rd floor marketing department, then deliver coffee to the 5th floor conference room” is input by voice input, or “Designate the route from the 3rd floor marketing department to the 5th floor conference room by dragging it on the map within the building” is input by GUI input, the robot control system (100) can preprocess the input draft scenario to generate a multi-modal draft scenario such as “Move to the 3rd floor marketing department and place the document, then move to the 5th floor conference room and place the coffee there.”

[0074] By utilizing two or more input methods in this way, users can effectively input draft scenarios for their needs.

[0075] Additionally, the robot control system (1000) can detect keywords related to robot control from the input draft scenario (S200). Here, keywords are words or phrases with significant meaning in the user-entered commands or scenarios, and can be used to clearly define the robot's task goals and methods. For example, keywords can be words indicating the type, location, target, or action of the task to be performed by the robot, such as "movement," "arrival," "cleaning," "delivery," or "security."

[0076] Specifically, when a user inputs a draft scenario using voice and text input, the draft scenario is input in natural language, so the scenario module (1200) of the robot control system (1000) can utilize natural language processing (NLP) technology to interpret the syntax of the input sentence, identify verbs and nouns indicating major actions, and extract keywords related to robot control. At this time, the robot control system (1000) can analyze the context of the sentence, process polysemous and complex sentences, and appropriately detect keywords.

[0077] Furthermore, the robot control system (1000) can recognize and compensate for various input variations and differences in the user's pronunciation, intonation, and expression style through continuous learning, thereby accurately detecting keywords. For example, even if the user inputs the command "Start food delivery from the kitchen" differently depending on the expression style, such as "Bring food to the kitchen" or "Deliver to the kitchen," the robot control system (1000) can interpret these as having the same meaning and accurately detect keywords such as "delivery" and "kitchen."

[0078] That is, the robot control system (1000) can detect keywords by recognizing new input methods or pronunciation patterns through adaptive learning.

[0079] In addition, when a user inputs a draft scenario using a GUI input method, when configuring the scenario, he or she is provided with options for settings, commands, etc. for the robot's operations, and each option is matched with a predefined keyword, and the robot control system (1000) can detect keywords related to robot control in the draft scenario through the matched keywords.

[0080] Additionally, when a user inputs a draft scenario in multiple input modes, the robot control system (1000) can detect keywords related to robot control from the multi-modal draft scenario.

[0081] Additionally, the robot control system (1000) can recommend the robot's main action and sub-action to the user based on the detected keywords (S300). Here, the robot's main action represents the main task performed by the robot, and the sub-action represents a detailed task that supports or assists the main action.

[0082] Specifically, a main action can consist of actions that represent the primary task or goal the robot must perform. For example, a main action can be comprised of essential parts of a task, such as "Move," "Arrive," "Start," "Pick up a tray," or "Return to a waiting area," and can define the goals the robot must achieve.

[0083] Additionally, sub-actions are subtasks that assist the main action and can be composed of detailed tasks to support the execution of the main action. These can be defined as necessary components to increase the efficiency and success rate of the main action, such as "BGM operation," "LED operation," "TTS playback," "tray check," and "battery check."

[0084] That is, the main action represents the main task or goal that the robot must perform, and the sub-action can mean a detailed sub-task to assist the main action and support its execution.

[0085] Additionally, the robot control system (1000) can recommend to the user the main action and sub-action of the robot that are matched by each detected keyword.

[0086] Additionally, the robot control system (1000) can select the robot's main and sub-actions and recommend them to the user by considering not only the detected keywords but also various data (e.g., robot data, environmental data, etc.). This will be further explained with reference to FIG. 5.

[0087] Figure 5 is a flowchart specifically illustrating a scenario generation method according to one embodiment of the present invention.

[0088] Referring to FIG. 5, the robot control system (1000) can select main actions and sub-actions corresponding to keywords (S310).

[0089] Specifically, the robot control system (1000) can analyze the context of a keyword among the main actions and sub-actions of a robot matched to each keyword, and select main actions and sub-actions related to robot control corresponding to the keyword. For example, the keyword "movement" may mean delivering a package in a "delivery" service scenario, but may mean the robot moving while removing dust in a "cleaning" service scenario. Therefore, the main actions and sub-actions can be selected based on the context of the keyword.

[0090] Here, the robot control system (1000) generates a syntax tree, a tree structure that visually shows how words constituting a sentence interact and are combined, to determine the context in which a keyword is used, thereby analyzing the role (subject, verb, object, etc.) each keyword plays within the sentence. In this case, the robot control system (1000) may be used in conjunction with word embedding and semantic role labeling technologies.

[0091] That is, the robot control system (1000) can analyze the context of a keyword and initially select main actions and sub-actions related to the control of a robot corresponding to the keyword.

[0092] Next, the robot control system (1000) can secondarily select main actions and sub-actions related to robot control based on environmental data about the location where the service is to be provided and robot data about the robot's functions (S320).

[0093] Here, environmental data may include location and spatial information (e.g., a map of the space the robot will work in, location information, whether movement between floors is required), obstacle information (e.g., the location of physical obstacles, available space and restricted areas), climate and lighting condition information (e.g., room temperature, humidity, lighting level), floor condition information (e.g., floor material (carpet, tile, etc.) and condition (wet, slippery)).

[0094] Additionally, robot data may include robot function and specification information (e.g., robot movement speed, maximum load weight, etc.), battery status information (e.g., current battery level, expected battery consumption, etc.), sensor status information (e.g., status and sensed information of each sensor such as camera, distance sensor, ultrasonic sensor, etc.), task history information (e.g., history such as success rate and failure cause of previous tasks), etc.

[0095] Specifically, the robot control system (1000) can input environmental data and robot data into a pre-trained artificial neural network model to select a combination of main actions and sub-actions (or action combination). At this time, the artificial neural network model can be trained to select an optimal combination of main actions and sub-actions based on the environmental data and robot data, and a reinforcement learning-based model, a recurrent neural network (RNN), a transformer model, etc. can be used.

[0096] For example, the robot control system (1000) can select a sub-action that activates LED operation in a dark environment through a pre-learned artificial neural network model, and can select a sub-action that deactivates LED operation to save energy in an environment with sufficient lighting.

[0097] As another example, the robot control system (1000) can select the main action and sub-action of the robot to minimize unnecessary sub-actions and focus on the main action by switching to energy saving mode when the remaining battery level is low enough to complete a service through a pre-learned artificial neural network model.

[0098] That is, the robot control system (1000) can secondarily select main actions and sub-actions related to the control of the first-selected robot based on environmental data and robot data.

[0099] Next, the robot control system (1000) can select main actions and sub-actions related to robot control based on user-specific selection data (S330). Here, user-specific selection data refers to various types of data collected based on the history of multiple users' interactions with the robot.

[0100] For example, user-specific selection data may include the frequency of the main action selected by each user to perform a specific task (number of main action selections), the frequency of the sub-actions selected by each user to match the main action selected to perform a specific task (number of sub-action selections), whether a specific combination of actions succeeds in performing a task (task success rate), feedback such as subjective evaluations or corrections provided by the user after performing a task (user feedback), and the time taken to perform each main action or sub-action (time required).

[0101]

[0102] Specifically, the robot control system (1000) can identify combinations of main actions and sub-actions frequently used by other users through statistical analysis based on the frequency of main actions and sub-actions in user-specific data. In this case, the statistical analysis focuses on identifying patterns of combinations that are commonly selected in various user environments.

[0103] In addition, the robot control system (1000) can select the main action and sub-action by considering the task success rate, user feedback, and required time of the identified combination of main actions and sub-actions.

[0104] For example, if the combination of “obstacle avoidance” and “turning on LED light” is frequently used in the “living room cleaning” task and has a high success rate and user satisfaction, the robot control system (1000) can select the combination of the main action and sub-action.

[0105] Additionally, the robot control system (1000) can select selection data of other users who have used services similar to the user's service and use it for tertiary selection.

[0106] As described above, the robot control system (1000) can select through a primary selection process based on keywords, a secondary selection process based on environmental data and robot data, and a tertiary selection process based on user-specific data, and recommend the selected main actions and sub-actions to the user.

[0107] In addition, the robot control system (1000) can recommend main actions and sub-actions to the user by determining internal parameters of the selected main actions and sub-actions based on user setting data or average setting data of other users. Here, the internal parameters refer to variables that adjust the detailed method and conditions of the corresponding action when performing a specific sub-action. For example, the internal parameters of the sub-action 'Add LED' may include LED color (white, blue, red, green, etc.), LED mode (lighting, blinking, etc.), LED position (front, back, etc.), the internal parameters of 'Rotation' may include 45 degrees, 90 degrees, 135 degrees, etc., and the internal parameters of 'Add BGM' may include BGM1, BGM 2, BGM 3, etc.

[0108] Additionally, the robot control system (1000) can determine the priority of selected main actions and sub-actions and recommend them to the user by considering the user's past usage patterns. For example, the robot control system (1000) can recommend to the user the main actions and sub-actions selected in order of greatest similarity to the main actions and sub-actions previously selected by the user.

[0109] Additionally, the robot control system (1000) can determine the priority of selected main and sub-actions based on the user's context and recommend them to the user. Here, "context" refers to the user's current situation or environment, and may include various factors such as time, location, activity, surroundings, device status, and the user's psychological state.

[0110] For example, the robot control system (1000) may recommend work-related main and sub-actions when in the office, and may recommend actions related to rest or housework when at home.

[0111] As another example, the robot control system (1000) may recommend a sub-action to turn on the lights and play background music in a low-light and quiet environment, and conversely, may recommend an action to reduce noise in a very noisy environment.

[0112] That is, the robot control system (1000) can determine the priority of main actions and sub-actions based on the user's location, time zone, surrounding environmental conditions, etc. and recommend them to the user.

[0113] In addition, the robot control system (1000) may provide a reason for the recommendation when recommending a main action and a sub-action through explainable AI (XAI). Here, the explainable AI (XAI) may be trained to clearly explain the basis for the recommendation by analyzing environmental data, robot data, and user-specific selection data. For example, the robot control system (1000) may generate a reason for the recommendation such as, "Based on the keyword 'delivery' requested by the user, 'document delivery' was identified as the most suitable main action as a result of related data analysis. This action achieved a user satisfaction rate of over 90% in similar requests," and provide the result to the user.

[0114] Referring again to FIG. 3, the robot control system (1000) can verify user approval for the recommended main and sub-actions (S400). Specifically, the robot control system (1000) considers approval complete when the user presses the "Final Confirmation" button. However, if the user selects only a portion of the actions or modifies internal parameters, the robot control system (1000) may consider approval disapproved.

[0115] If the user does not approve (S400, NO), the robot control system (1000) may receive the user's modified scenario for the service (S600). Here, the modified scenario refers to the robot's work plan updated based on the draft scenario to accommodate changes in user needs or environmental conditions. The modified scenario may also refer to a step that optimizes or refines the robot's action plan by reflecting additional user input or feedback based on the work goals and conditions defined in the draft scenario.

[0116] These revised scenarios can be entered using the same input method as the draft scenario entry step (S100) described above. For example, revised scenarios can be entered that include changing the order of recommended main actions and sub-actions, modifying internal parameters of sub-actions, or selecting other main actions or sub-actions.

[0117] In addition, the robot control system (1000) can re-detect keywords in the input modification scenario (S200) and re-recommend main actions and sub-actions (S300) based on the re-detected keywords. At this time, the keyword re-detection step (S200) may be omitted depending on the input modification scenario.

[0118] That is, the above-described keyword detection step (S200) and recommendation step (S300) can be repeatedly performed according to user approval.

[0119] Meanwhile, if the user approves (S400, NO), the robot control system (1000) can generate a service scenario based on the determined main action and sub-action (S500).

[0120] Specifically, the robot control system (1000) can generate a service scenario by integrating the robot's action tree, task plan, and assignment based on the determined main and sub-actions. Here, the service scenario is a final robot task plan tailored to the user's needs and environmental conditions, and represents a comprehensive plan for the robot's task performance by integrating the robot's action tree, task plan, and assignment.

[0121] That is, the robot control system (1000) can create or modify a scenario for a service desired by the user based on user approval.

[0122] According to the present invention described above, the scenario creation process can be drastically shortened by recommending main actions and sub-actions of a scenario for robot control based on keywords in a scenario input by a user.

[0123] Additionally, according to the present invention, a user can input an overall scenario of a service to be provided by the robot through various input methods.

[0124] In addition, the present invention provides a graphical user interface that allows a user to directly design a scenario using drag and drop, thereby providing an environment that is easily accessible even to non-experts.

[0125]

[0126] Meanwhile, the robot control system (1000) can learn the user's tendencies through the user's approval process, and this will be further described with reference to FIG. 6.

[0127] Figure 6 is a flowchart illustrating a scenario creation method according to one embodiment of the present invention.

[0128] Referring to FIG. 6, the robot control system (1000) can learn the user's tendency to select or set the main action and sub-action depending on the user's approval.

[0129] Specifically, the robot control system (1000) analyzes the user's pattern of approving or modifying specific tasks, thereby learning the user's preferences and behavioral patterns, which can then be reflected in future main and sub-action recommendations. In this case, an artificial neural network model can be used to learn the user's tendencies and reflect them in the recommendation process.

[0130] For example, if a user repeatedly selects a specific sub-action or adjusts internal parameters, the robot control system (1000) can learn this and automatically reflect this preference in future similar task scenarios to recommend main actions and sub-actions.

[0131] In this way, the present invention can continuously improve the user experience by providing personalized recommendations when creating future scenarios, thereby increasing the consistency of robot control and user satisfaction.

[0132]

[0133] Next, the graphical user interface used in the GUI input method among the scenario input methods of the present invention can be implemented in a drag-and-drop manner for the scenario, and this will be described with reference to FIG. 7.

[0134] FIG. 7 is an exemplary diagram showing a graphical user interface according to one embodiment of the present invention.

[0135] Referring to FIG. 7, the graphical user interface may be composed of a first area (71) displaying a list of user scenarios, a second area (72) displaying a list of sub-actions, a third area (73) displaying the main action of the scenario selected in the first area, and a fourth area (74) displaying the sub-actions of the scenario selected in the first area.

[0136] Specifically, the user can select an existing scenario in the first area (71) and modify the selected scenario or add a new scenario.

[0137] Additionally, the user can change the order or type of the main actions by dragging and dropping the recommended main actions or existing main actions through the third area (73), clicking a button, etc.

[0138] Additionally, the user can change the order or type of sub-actions by dragging and dropping the recommended main action or sub-action corresponding to an existing main action through the fourth area (74), and can modify the internal parameters of the sub-actions.

[0139] Additionally, the user can select at least one of the list of sub-actions in the second area (72) and add it to the fourth area (74) by dragging and dropping.

[0140] This graphical user interface is an example of one of the input methods of the present invention and may be modified into other forms and methods as needed.

[0141] Meanwhile, the robot control system (1000) of the present invention or each module within the robot control system (1000) can be implemented as a computing device, and this will be described with reference to FIG. 8.

[0142] Figure 8 is an exemplary diagram showing the configuration of a computing device according to one embodiment of the present invention.

[0143] Referring to FIG. 8, in some embodiments of the present invention, the robot control system (1000) may be implemented in the form of a computing device.

[0144] At least one of each module constituting the robot control system (1000) is implemented on a general-purpose computing processor and thus may include a processor (1008), an input / output I / O (1002), a memory (1004), an interface (1006), and a bus (1014). The processor (1008), the input / output device (1002), the memory (1004), and / or the interface (1006) may be coupled to each other via the bus (1014). The bus (1014) corresponds to a path through which data is transferred.

[0145] Specifically, the processor (1008) may include at least one of a Central Processing Unit (CPU), a Micro Processor Unit (MPU), a Micro Controller Unit (MCU), a Graphic Processing Unit (GPU), a microprocessor, a digital signal processor, a microcontroller, an application processor (AP), and logic elements capable of performing functions similar thereto.

[0146] The input / output device (1002) may include at least one of a keypad, a keyboard, a touchscreen, and a display device. The memory (1004) may store data and / or programs.

[0147] The interface (1006) may perform a function of transmitting data to or receiving data from a communication network. The interface (1006) may be wired or wireless. For example, the interface (1006) may include an antenna or a wired / wireless transceiver. The memory (1004) may further include high-speed DRAM and / or SRAM, etc., as a volatile operating memory that enhances the operation of the processor (1008) while protecting personal information.

[0148] Additionally, the memory (1004) stores programming and data configurations that provide the functionality of some or all of the modules described herein. For example, it may include logic for performing selected aspects of the scenario generation method according to the present embodiment.

[0149] A program or application is loaded with a set of commands including each operation of the present invention described above stored in the memory (1004), and the processor can perform each operation. Here, each operation may include an operation of receiving a draft scenario for a service to be performed by a robot from a user, an operation of detecting keywords related to robot control from the input draft scenario, an operation of recommending the main action and sub-action of the robot to the user based on the detected keywords, an operation of generating a scenario for a service based on user approval of the recommendation, and an operation of receiving a modified scenario for a service based on user approval of the recommendation.

[0150] The various embodiments described herein may be implemented in a recording medium readable by a computer or similar device, for example, using software, hardware, or a combination thereof.

[0151] In terms of hardware implementation, the embodiments described herein can be implemented using at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors, controllers, micro-controllers, microprocessors, and other electrical units for performing functions. In some cases, the embodiments described herein can be implemented as a control module itself.

[0152] In a software implementation, the procedures and functions described herein, as well as other embodiments, may be implemented as separate software modules. Each of these software modules may perform one or more of the functions and operations described herein. The software code may be implemented as a software application written in a suitable programming language. The software code may be stored in a memory module and executed by a control module.

[0153] The above description is merely an example of the technical idea of ​​the present invention, and those skilled in the art will appreciate that various modifications, changes, and substitutions can be made without departing from the essential characteristics of the present invention.

[0154] Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate, rather than limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by these embodiments and the accompanying drawings. The protection scope of the present invention should be interpreted by the following claims, and all technical concepts within the scope equivalent thereto should be interpreted as being included within the scope of the rights of the present invention.

Claims

1. A method for generating a scenario for robot control performed on a computing device, A step of receiving a draft scenario for a service to be performed by the robot from the user; A step of detecting keywords related to robot control from an input draft scenario; A step of recommending the main action and sub-action of the robot to the user based on the detected keywords; A method for generating a scenario, comprising: a step of generating a scenario for the service based on user approval for the above recommendation; 2. In paragraph 1, Further comprising a step of receiving a modification scenario for the service based on user approval for the above recommendation, A method for generating a scenario, wherein the step of detecting the keyword comprises re-detecting the keyword in the modified scenario.

3. In paragraph 1, A method for generating a scenario, characterized in that the above recommended steps are repeatedly performed according to the user's approval.

4. In paragraph 1, A method for generating a scenario, characterized in that the above-mentioned recommended step analyzes the context of the above-mentioned keyword and first selects main actions and sub-actions related to the control of the robot corresponding to the keyword.

5. In paragraph 4, A method for generating a scenario, characterized in that the above-mentioned recommended step secondarily selects main actions and sub-actions related to the control of a robot that has been firstly selected based on environmental data about a place where the service is to be provided and robot data about the function of the robot.

6. In paragraph 5, A method for generating a scenario, characterized in that the above recommended step tertiarily selects main actions and sub-actions related to the control of a robot that has been secondarily selected based on user-specific selection data.

7. In paragraph 5, A method for generating a scenario, wherein the user-specific selection data includes the selection frequency of main actions and sub-actions selected by other users.

8. Processor, and including a memory communicating with the processor, The above memory stores instructions that cause the processor to perform operations, The above actions are actions of receiving a draft scenario for a service to be performed by the robot from the user; An action of detecting keywords related to robot control from an input draft scenario; An action of recommending the main action and sub-action of the robot to the user based on the detected keywords; A computing device comprising: an operation for generating a scenario for the service based on user approval of the above recommendation; 9. In paragraph 8, Further comprising an action of receiving a modification scenario for the service based on user approval for the above recommendation; A computing device characterized in that the action of detecting the keyword comprises re-detecting the keyword in the modification scenario.

10. In paragraph 8, A computing device characterized in that the above recommended action analyzes the context of the keyword and first selects main actions and sub-actions related to the control of the robot corresponding to the keyword.

11. In paragraph 10, A computing device characterized in that the above recommended action secondarily selects main actions and sub-actions related to the control of a robot that has been first selected based on environmental data about a place where the service is to be provided and robot data about the function of the robot.

12. In paragraph 11, A computing device characterized in that the above recommended action is a third-stage selection of main actions and sub-actions related to the control of a second-stage selected robot based on user-specific selection data.

13. In paragraph 11, A computing device, characterized in that the user-specific selection data includes the selection frequency of main actions and sub-actions selected by other users.

14. A computer-readable recording medium storing a program for performing a scenario generation method according to any one of paragraphs 1 to 7.

15. A program stored on a computer-readable recording medium including a program code for executing a scenario generation method according to any one of paragraphs 1 to 7.

Images