Method, computing device, and computer program for verifying service scenario by using simulation of virtual environment

Abstract

The present invention relates to a method for verifying a service scenario by using a simulation of a virtual environment, and may comprise the steps of: generating a virtual environment corresponding to a real environment in which a service scenario is to be executed; simulating an operation of a robot on the basis of the service scenario in the generated virtual environment; and verifying the service scenario by using result data of the simulation. Acknowledgement: This patent is the result of research conducted, with the support of the Korea Institute of Procurement, using funds of the government (public procurement service) in 2025. (No.RS-2025-16072908).

Description

Method for verifying service scenarios through simulation of a virtual environment, computing device and computer program

[0001] The present invention relates to a method for verifying a service scenario through simulation of a virtual environment, a computing device, and a computer program.

[0002] Recently, various types of robots are being used in real life and factory automation.

[0003] These robots have specialized control scenarios tailored to their specific purposes. In factory automation, motion control scenarios suited to repetitive tasks are primarily used, and these scenarios are designed to allow robots to perform repetitive tasks to maximize efficiency.

[0004] However, simple repetitive scenarios are not enough for service robots used in daily life or in various places.

[0005] For example, guide robots must interact with users and provide information in real time, and delivery robots must avoid obstacles in their path and safely reach their destination.

[0006] Additionally, even for the same robot, the scenario for robot control often needs to be modified depending on the location where it is used or the service provided, so the robot must be controlled in a complex environment.

[0007] Simulation is necessary to verify these complex scenarios. Simulation allows for preliminary testing of how robots will operate in various environments, allowing for the early detection and correction of problems. However, conventional methods have encountered numerous problems due to insufficient sophistication in the process of verifying robot scenarios.

[0008] Conventional robot simulation methods primarily operate in static environments and fail to adequately reflect the complexities and dynamic variables of the real world. For example, they often fail to predict how robots will respond to specific obstacles or changing environmental conditions. Furthermore, existing simulation methods often deviate significantly from real-world environments, resulting in simulation results that often fail to accurately reflect actual robot performance.

[0009] This has often resulted in service robots encountering unexpected problems in real-world environments, which can reduce the reliability and efficiency of services.

[0010] Therefore, a new method to pre-validate and optimize service scenarios became necessary.

[0011] The purpose of the present invention is to propose a method for verifying a service scenario through simulation of a virtual environment, a computing device, and a computer program.

[0012] The objectives of the present invention are not limited to those mentioned above. Other objectives and advantages of the present invention not mentioned above can be understood through the following description and will be more clearly understood through the embodiments of the present invention. Furthermore, it will be readily apparent that the objectives and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims.

[0013] In order to achieve the above-described purpose, a method for verifying a service scenario through a virtual environment simulation performed in a computing device according to an embodiment of the present invention may include a step of creating a virtual environment corresponding to an actual environment in which a service scenario is to be executed, a step of simulating the operation of a robot based on the service scenario in the created virtual environment, and a step of verifying the service scenario using result data of the simulation.

[0014] Additionally, the simulating step may include simulating by adding intended variable situations to the virtual environment.

[0015] Additionally, the above variable situation may include at least one of a sensor malfunction of the robot, a change in battery status, and a network connection problem.

[0016] In addition, the step of verifying the above service scenario can verify the service scenario by evaluating whether the service scenario can achieve the expected target standard in an actual environment based on the above result data.

[0017] Additionally, the above target criteria may include the task success rate, time efficiency, energy consumption, and safety of the robot.

[0018] In addition, a step of generating learning data based on the robot's behavioral data corresponding to the above variable situation may be further included.

[0019] Meanwhile, the present invention is implemented as a computing computer, including a processor and a memory communicating with the processor, the memory storing instructions causing the processor to perform operations, and the operations may include an operation of creating a virtual environment corresponding to an actual environment in which a service scenario is to be executed, an operation of simulating the operation of a robot based on the service scenario in the created virtual environment, and an operation of verifying the service scenario using result data of the simulation.

[0020] Additionally, the above-mentioned simulated action can be simulated by adding intended variable situations to the above-mentioned virtual environment.

[0021] Additionally, the above variable situation may include at least one of a sensor malfunction of the robot, a change in battery status, and a network connection problem.

[0022] In addition, the operation of verifying the above service scenario can verify the service scenario by evaluating whether the service scenario can achieve the expected target standard in an actual environment based on the above result data.

[0023] Additionally, the above target criteria may include the task success rate, time efficiency, energy consumption, and safety of the robot.

[0024] In addition, it may further include an operation of generating learning data based on the robot's behavioral data corresponding to the above variable situation.

[0025] According to the present invention, by creating a virtual environment reflecting an actual environment, it is possible to test and verify in advance the tasks to be performed by a robot.

[0026] In addition, the present invention can discover and resolve unexpected problems in real environments in advance by verifying and optimizing complex scenarios of service robots through simulation of a virtual environment.

[0027] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0028] Figure 1 is a schematic diagram showing the configuration of a robot control system according to one embodiment of the present invention.

[0029] Figure 2 is an exemplary diagram showing the configuration of a robot control system according to one embodiment of the present invention.

[0030] Figure 3 is a flowchart illustrating a scenario creation method according to one embodiment of the present invention.

[0031] FIG. 4 is a flowchart illustrating a service scenario verification method through virtual environment simulation according to one embodiment of the present invention.

[0032] Figure 5 is an exemplary diagram showing a virtual environment according to one embodiment of the present invention.

[0033] Figure 6 is a flowchart illustrating a service scenario verification method according to one embodiment of the present invention in more detail.

[0034] FIG. 7 and FIG. 8 are exemplary diagrams showing a user interface for simulation according to one embodiment of the present invention.

[0035] Figure 9 is an exemplary diagram showing the configuration of a computing device according to one embodiment of the present invention.

[0036] The following merely illustrates the principles of the present invention. Therefore, those skilled in the art will be able to implement the principles of the present invention and invent various devices within the scope and spirit of the present invention, even if not explicitly described or illustrated herein. Furthermore, all conditional terms and embodiments listed herein are expressly intended, in principle, to facilitate understanding of the concepts of the present invention, and should be understood as being in no way limiting to the specifically enumerated embodiments and conditions.

[0037] The above-described objects, features and advantages will become more apparent through the following detailed description with reference to the attached drawings, so that a person having ordinary skill in the art to which the present invention pertains can easily practice the technical idea of ​​the present invention.

[0038] In addition, in describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

[0039] Hereinafter, various embodiments of the present invention will be described in detail with reference to the attached drawings.

[0040] Figure 1 is a schematic diagram showing the configuration of a robot control system according to one embodiment of the present invention.

[0041] Referring to FIG. 1, a robot control system (1000) according to the present invention may include a process for dynamically planning and optimizing the actions of a robot, and the robot control system (1000) may provide a robot control service aimed at automatically generating and coordinating the tasks of the robot through various steps, such as inputting scenarios / commands for robot control, action planning, and execution management. Here, a robot may refer to any type of robot that can automate specific tasks, extend human capabilities, and perform tasks in difficult-to-access locations.

[0042] For example, robots may include robots that perform exploration activities in difficult-to-access environments, robots that assist with precise surgery or treatment in the medical field, and robots that perform tasks such as assembling, packaging, welding, and painting products in manufacturing processes. In the present invention, the description will be based on robots that perform tasks such as cleaning, guiding, and delivery for the purpose of serving general consumers or providing services.

[0043] The robot control service of the robot control system (1000) can be broadly divided into three parts: a scenario process for inputting, outputting, generating, and managing scenarios for robot control; a work process for planning and assigning robot behavior trees (BTs) and tasks; and a simulation process for executing simulations. Each process can interact and be operated in an integrated manner to optimally achieve the user's desired results.

[0044] First, regarding the scenario process, a user using the robot control system (1000) can input scenarios or control commands for tasks to be performed by the robot through an app (APP, 11) that provides robot control services. The user-entered scenarios can be easily managed (edited, created, etc.) through an intuitive user interface and adjusted to various work environments and purposes. In this case, the user may refer to an employee (e.g., an on-site consulting employee) who contacts a customer to provide robot control services, or a customer who utilizes the robot control services.

[0045] The user's input scenario is created as a scenario for a service through several modification processes via the scenario module (1200) in the robot control system (1000), and the created scenario can be reflected in the robot's work plan and allocation through the work creation module (1400).

[0046] And, through the simulation module (1300), the service scenario can be simulated as a virtual robot in a virtual environment (12) and then distributed as a robot that will provide the service desired by the user.

[0047]

[0048] Hereinafter, the overall operation of the robot control system (1000) will be described with reference to FIG. 2.

[0049] Figure 2 is an exemplary diagram showing the configuration of a robot control system according to one embodiment of the present invention.

[0050] Referring to FIG. 2, a user (21) can provide user input to a robot control system (1000) to create a scenario or manage (e.g., modify) an existing scenario through an interface module (1100) for robot control suitable for a desired service. Here, the interface module (1100) can interact with the user through a web browser or an app.

[0051] In addition, the scenario module (1200) can create a new scenario or modify an existing scenario based on user input entered through the interface module (1100), and can manage / modify various items related to the scenario (e.g., movement path, map, BGM, etc.).

[0052] Next, the service scenario generated in the scenario module (1200) and various data related to the service scenario can be provided to the task generation module (1400) via the data management module (1500). Here, the data management module (1500) is responsible for service management and data relay management, and can coordinate smooth data flow and tasks with other modules of the robot control system (1000).

[0053] In addition, the task creation module (1400) establishes task allocation and task plans for each robot using a service scenario and user input, creates a robot behavior tree (BT) and reflects it in the user's scenario to create a service scenario, and this service scenario can be provided to the simulation module (1300) via the data management module (1500).

[0054] Additionally, the simulation module (1300) performs services using a virtual robot in a virtual environment based on a service scenario, thereby verifying (or testing) the service scenario and acquiring various data (e.g., success or failure, spatial status, learning data, etc.). The various data acquired in this way are provided to the task generation module (1400) and can be used to optimize the service scenario.

[0055] In addition, the data management module (1500) can distribute service scenarios to a robot (22) via a relay module (1600). Here, the relay module (1600) is a module that supports safe and efficient data communication in a network environment and can perform packet relay and TLS mutual authentication.

[0056] Additionally, the data storage module (1700) can store various data generated or input during the operation of the robot control system (1000).

[0057] For example, the data storage module (1700) may store performance data, reference data, simulator data, environmental data, robot data, sensing data, service data, keywords matching main actions and sub-actions, etc.

[0058] Through each module of the robot control system (1000) described above, a user can create and modify various desired scenarios in real time, and by verifying and optimizing the proposed scenarios through simulation, a robot control service that can quickly respond to dynamic environmental changes can be provided.

[0059]

[0060] Next, a method for generating a scenario for robot control in a robot control system (1000) is described with reference to FIG. 3.

[0061] Figure 3 is a flowchart illustrating a scenario creation method according to one embodiment of the present invention.

[0062] Referring to FIG. 3, the robot control system (1000) can receive a draft scenario for a service to be performed by the robot from the user (S100). Here, the draft scenario is an initial input stage in the robot control system (1000) where the user defines the basic plan and requirements for the task to be performed by the robot. The user can set the robot's task goals and environmental conditions and provide an outline of the task to be performed by the robot. For example, the draft scenario may include information about the path, movements, etc. for the service to be performed by the robot.

[0063] Specifically, the interface module (1100) of the robot control system (100) supports various input methods such as a GUI (Graphical User Interface) input method, a voice input method, and a text input method so that a user can input a scenario intuitively and efficiently, and each input method will be described.

[0064] First, let's explain the GUI input method. Users can input scenarios through an intuitive graphical user interface. The GUI input method allows users to interact with the robot through drag-and-drop, button clicks, and other methods, allowing them to easily configure the robot's work path, target points, and task details.

[0065] For example, a user can specify a robot's movement path by dragging it on an on-screen map, and enter a draft scenario by selecting and adding the required tasks from a task list.

[0066] Next, the text input method is described. A user can input a scenario in text format. At this time, the robot control system (100) can recommend commands or settings for controlling the robot based on the text entered by the user, and the user can select the recommended commands or settings to input a draft scenario. This text input can be useful for accurately conveying detailed commands or settings.

[0067] Additionally, when explaining the voice input method, users can input draft scenarios using natural language. For example, a user can input a draft scenario for the robot by voice-referencing a command such as "Start cleaning in the living room at 10 o'clock and move to the kitchen after cleaning."

[0068] Meanwhile, the draft scenario input into the robot control system (1000) may be input by a consulting staff member who conducts consultation with a customer receiving the service.

[0069] Additionally, the robot control system (100) can receive a draft scenario from at least two input methods among a GUI (Graphical User Interface) input method, a voice input method, and a text input method.

[0070] In this case, the robot control system (100) can preprocess each draft scenario input through multiple input methods to generate a multi-modal draft scenario. Here, the multi-modal draft scenario may refer to a single comprehensive draft scenario that combines data collected through multiple input methods.

[0071] Specifically, the robot control system (100) can generate a multi-modal draft scenario in text format by organizing content inconsistencies or duplicate information between various input formats or resolving conflicts in draft scenarios input through various input methods.

[0072] For example, if a draft scenario such as “Deliver the document to the 3rd floor marketing department, then deliver coffee to the 5th floor conference room” is input by voice input, or “Designate the route from the 3rd floor marketing department to the 5th floor conference room by dragging it on the map within the building” is input by GUI input, the robot control system (100) can preprocess the input draft scenario to generate a multi-modal draft scenario such as “Move to the 3rd floor marketing department and place the document, then move to the 5th floor conference room and place the coffee there.”

[0073] By utilizing two or more input methods in this way, users can effectively input draft scenarios for their needs.

[0074] Additionally, the robot control system (1000) can detect keywords related to robot control from the input draft scenario (S200). Here, keywords are words or phrases with significant meaning in the user-entered commands or scenarios, and can be used to clearly define the robot's task goals and methods. For example, keywords can be words indicating the type, location, target, or action of the task to be performed by the robot, such as "movement," "arrival," "cleaning," "delivery," or "security."

[0075] Specifically, when a user inputs a draft scenario using voice and text input, the draft scenario is input in natural language, so the scenario module (1200) of the robot control system (1000) can utilize natural language processing (NLP) technology to interpret the syntax of the input sentence, identify verbs and nouns indicating major actions, and extract keywords related to robot control. At this time, the robot control system (1000) can analyze the context of the sentence, process polysemous and complex sentences, and appropriately detect keywords.

[0076] Furthermore, the robot control system (1000) can recognize and compensate for various input variations and differences in the user's pronunciation, intonation, and expression style through continuous learning, thereby accurately detecting keywords. For example, even if the user inputs the command "Start food delivery from the kitchen" differently depending on the expression style, such as "Bring food to the kitchen" or "Deliver to the kitchen," the robot control system (1000) can interpret these as having the same meaning and accurately detect keywords such as "delivery" and "kitchen."

[0077] That is, the robot control system (1000) can detect keywords by recognizing new input methods or pronunciation patterns through adaptive learning.

[0078] In addition, when a user inputs a draft scenario using a GUI input method, when configuring the scenario, he or she is provided with options for settings, commands, etc. for the robot's operations, and each option is matched with a predefined keyword, and the robot control system (1000) can detect keywords related to robot control in the draft scenario through the matched keywords.

[0079] Additionally, when a user inputs a draft scenario in multiple input modes, the robot control system (1000) can detect keywords related to robot control from the multi-modal draft scenario.

[0080] Additionally, the robot control system (1000) can recommend the robot's main action and sub-action to the user based on the detected keywords (S300). Here, the robot's main action represents the main task performed by the robot, and the sub-action represents a detailed task that supports or assists the main action.

[0081] Specifically, a main action can consist of actions that represent the primary task or goal the robot must perform. For example, a main action can be comprised of essential parts of a task, such as "Move," "Arrive," "Start," "Pick up a tray," or "Return to a waiting area," and can define the goals the robot must achieve.

[0082] Additionally, sub-actions are subtasks that assist the main action and can be composed of detailed tasks to support the execution of the main action. These can be defined as necessary components to increase the efficiency and success rate of the main action, such as "BGM operation," "LED operation," "TTS playback," "tray check," and "battery check."

[0083] That is, the main action represents the main task or goal that the robot must perform, and the sub-action can mean a detailed sub-task to assist the main action and support its execution.

[0084] Additionally, the robot control system (1000) can recommend to the user the main action and sub-action of the robot that are matched by each detected keyword.

[0085] In addition, the robot control system (1000) can select the main action and sub-action of the robot and recommend them to the user by considering not only the detected keywords but also various data (e.g., robot data, environmental data, etc.).

[0086] Next, the robot control system (1000) can verify user approval for the recommended main and sub-actions (S400). Specifically, the robot control system (1000) considers approval complete when the user presses the "Final Confirmation" button. However, if the user selects only a portion of the actions or modifies internal parameters, the robot control system (1000) may consider approval disapproved.

[0087] If the user does not approve (S400, NO), the robot control system (1000) may receive the user's modified scenario for the service (S600). Here, the modified scenario refers to the robot's work plan updated based on the draft scenario to accommodate changes in user needs or environmental conditions. The modified scenario may also refer to a step that optimizes or refines the robot's action plan by reflecting additional user input or feedback based on the work goals and conditions defined in the draft scenario.

[0088] These revised scenarios can be entered using the same input method as the draft scenario entry step (S100) described above. For example, revised scenarios can be entered that include changing the order of recommended main actions and sub-actions, modifying internal parameters of sub-actions, or selecting other main actions or sub-actions.

[0089] In addition, the robot control system (1000) can re-detect keywords in the input modification scenario (S200) and re-recommend main actions and sub-actions (S300) based on the re-detected keywords. At this time, the keyword re-detection step (S200) may be omitted depending on the input modification scenario.

[0090] That is, the above-described keyword detection step (S200) and recommendation step (S300) can be repeatedly performed according to user approval.

[0091] Meanwhile, if the user approves (S400, NO), the robot control system (1000) can generate a service scenario based on the determined main action and sub-action (S500).

[0092] Specifically, the task generation module (1400) of the robot control system (1000) can allocate and plan tasks for the robot based on the determined main actions and sub-actions of the scenario, and generate a behavior tree to generate a service scenario. Here, the service scenario is a final robot task plan adjusted to the user's needs and environmental conditions, and means a comprehensive plan for the robot's task performance by integrating the robot's behavior tree, task planning, and assignment.

[0093] The service scenarios generated in this way can be distributed to each robot, and can also undergo a verification process through simulation before distribution.

[0094] Hereinafter, with reference to Fig. 4, a method for verifying a service scenario through virtual environment simulation is described.

[0095] FIG. 4 is a flowchart illustrating a service scenario verification method through virtual environment simulation according to one embodiment of the present invention.

[0096] Referring to FIG. 4, the robot control system (1000) can create a virtual environment (S1000) corresponding to the actual environment in which the service scenario will be executed. Here, the virtual environment (Virtual Space) refers to a digitally created space that mimics the actual environment. The virtual environment reflects various characteristics, conditions, and elements of the physical world, enabling preliminary testing and verification of tasks that the robot will perform in the actual environment.

[0097] Specifically, the simulation module (1300) of the robot control system (1000) can obtain environmental data on the actual environment in which the service scenario will be executed from the data storage module (1700) or the data management module (1500), and can create a virtual environment based on the obtained environmental data. Here, the environmental data can include location and spatial information (e.g., a map of the space in which the robot will work, location information, whether interfloor movement is required), obstacle information (e.g., the location of physical obstacles, movable space and restricted areas), climate and lighting condition information (e.g., indoor temperature, humidity, lighting brightness level), floor condition information (e.g., floor material (carpet, tile, etc.) and condition (wetness, slipperiness)), etc.

[0098] For example, as shown in FIG. 5, the robot control system (1000) can create a virtual environment corresponding to environmental data, and can place static elements such as walls, pillars, and fixed obstacles and dynamic elements moving within the environment such as people and vehicles in the created virtual environment, and can create a virtual environment by reflecting environmental conditions such as lighting brightness and climate conditions.

[0099] In addition, the robot control system (1000) can implement a virtual environment by managing static and dynamic elements that constitute the virtual environment as separate layers, each consisting of a static layer and a dynamic layer. For example, a virtual environment can be constructed by overlaying various dynamic layers (e.g., obstacles, people, vehicles, etc.) on top of a basic static layer (e.g., buildings, roads, etc.), and unnecessary layers can be selectively deactivated depending on the situation. This simplifies the process. Through this, the robot control system (1000) can maintain the overall simulation as lightweight and fast as possible.

[0100] Next, the robot control system (1000) can simulate the robot's operation based on the service scenario in the generated virtual environment (S2000).

[0101] Specifically, the simulation module (1300) of the robot control system (1000) can obtain a service scenario from the data management module (1500) and simulate the operation of the robot in a virtual environment based on the service scenario and the robot data of the robots. Here, the robot data is data obtained from the robots in real time, and the robot data may include robot function and specification information (e.g., robot moving speed, maximum load weight, etc.), location information (e.g., current location, etc.), battery status information (e.g., current battery level, expected battery consumption, etc.), sensor status information (e.g., status and sensed information of each sensor such as a camera, distance sensor, ultrasonic sensor), and task history information (e.g., current task status, success rate of previous tasks, history such as cause of failure, etc.).

[0102] For example, the robot control system (1000) can simulate the robot moving along a specified path based on a service scenario, avoiding obstacles that may occur during movement, optimizing the path, etc., and simulating the robot picking up an object at a specific location or delivering it to a specific point.

[0103] In addition, the robot control system (1000) can perform a simulation by adding intended variable situations to a virtual environment. Here, the variable situations are various unexpected situations that may occur in an actual environment, and may include robot sensor malfunctions (e.g., failure of a camera sensor or ultrasonic sensor), battery status changes (e.g., rapid battery consumption, poor charging, etc.), network connection problems (e.g., network delay, intermittent disconnection, etc.), weather changes (e.g., sudden rain or snow, rapid temperature change, etc.), appearance of obstacles, software errors (e.g., data processing errors, command interpretation errors, etc.), and component failures.

[0104] Additionally, the robot control system (1000) can add variable situations by considering the interactivity and dependency between variable situations. Here, the interactivity between variable situations refers to a relationship in which a change in one variable situation affects other variable situations, and the dependency between variable situations refers to a relationship in which one variable situation changes depending on another variable situation. For example, since a network connection problem or sensor malfunction is likely to occur when the battery status is rapidly depleted, a rapid battery depletion situation and a network connection problem situation can be added as variable situations to perform a simulation. In this way, a more sophisticated simulation environment can be constructed through simulation that reflects the interactivity and dependency between multiple variable situations, and a robot can be simulated in a virtual environment more similar to the real environment.

[0105] Additionally, the robot control system (1000) may add additional variable situations in real time according to the robot's actions in response to variable situations while performing a simulation.

[0106] That is, the robot control system (1000) can reflect variable situations in a virtual environment to simulate how the robot responds to the variables.

[0107] Additionally, the robot control system (1000) can load specific objects or data elements only when necessary within the virtual environment during a simulation. For example, the robot control system (1000) can immediately load only objects within the robot's field of view, and not load the remaining objects until the robot approaches.

[0108] That is, the robot control system (100) can perform lazy loading of data within a virtual environment, thereby reducing memory usage and shortening the initial loading time.

[0109] Furthermore, the robot control system (1000) can prioritize individual objects or data within the virtual environment, loading critical elements first during the simulation. This allows less critical elements to be processed at a lower priority, preventing performance degradation and maximizing simulation efficiency.

[0110] In addition, the robot control system (1000) can simulate the operation of the robot based on the service scenario in the generated virtual environment and obtain result data for the simulation. Here, the result data may include the robot's task success rate and failure rate, path optimization, energy consumption (e.g., energy consumption per task, etc.), task execution time (e.g., time required per task, total time required, delay time), path optimization index (e.g., deviation between the actual movement path and the planned path), obstacle avoidance performance, sensor accuracy, interaction log (e.g., reaction time and response performance of the robot to environmental changes or specific events, etc.), task stability, etc.

[0111] Meanwhile, the robot control system (1000) can provide real-time feedback based on the resulting data generated during the simulation, allowing it to be reflected in robot data or service scenarios. For example, if the robot's battery depletes faster than expected, the path can be readjusted or the task order can be changed. This real-time feedback allows for a rapid response to unexpected situations, while simultaneously collecting data from various situations to further enhance future responsiveness.

[0112] Next, the robot control system (1000) can verify the service scenario using the simulation result data (S3000).

[0113] Specifically, the service scenario can be verified by evaluating whether the service scenario can achieve expected target criteria in an actual environment based on result data through the simulation module (1300) of the robot control system (1000). Here, the target criteria may include the robot's task success rate, time efficiency, energy consumption, and safety, and the reference values ​​of the target criteria may be variably determined depending on the work environment, user needs, task characteristics, target performance, and safety regulations.

[0114] Additionally, the robot control system (1000) can distribute the service scenario to each robot based on the service scenario verification results, or re-establish the robot's task allocation and planning for the service scenario. For example, if the service scenario verification results exceed a reference point, the robot control system (1000) can distribute the service scenario to each corresponding robot, and if the service scenario verification results are below the reference point, the robot control system (1000) can re-establish the robot's task allocation and planning for the scenario.

[0115] Meanwhile, the robot control system (1000) can generate learning data based on data acquired through simulation. This will be described with reference to FIG. 6.

[0116] Figure 6 is a flowchart illustrating a service scenario verification method according to one embodiment of the present invention in more detail.

[0117] Referring to FIG. 6, the robot control system (1000) can generate learning data based on the robot's behavioral data corresponding to variable situations (S4000).

[0118] Specifically, the simulation module (1300) of the robot control system (1000) can acquire behavioral data about the robot while simulating the robot's movements, including various variable situations, in a virtual environment. Here, the behavioral data refers to data collected based on how the robot behaved or what decisions it made when faced with various variable situations.

[0119] For example, behavioral data may include decision-making process data about what decisions a robot made in a variable situation, battery usage of the robot in a variable situation, the path the robot took in a variable situation and the speed at which it moved along that path, whether the robot successfully performed a given task in a variable situation, sensor data collected in a variable situation, etc.

[0120] Next, among the acquired behavioral data, behavioral data that causes incorrect judgments or shows poor performance in the decision-making process of the robot can be excluded, and successful, efficient, and adaptive behavioral data can be selected, and the selected behavioral data can be generated as a single learning data.

[0121] Next, the robot control system (1000) can re-establish the robot's task allocation and plan based on the learning data (S5000).

[0122] Specifically, the task generation module (1400) of the robot control system (1000) can re-assign tasks to robots and establish plans based on the learning data generated by the simulation module (1300). At this time, the robot control system (1000) can allocate tasks based on the learning data according to the performance and characteristics of each robot, or adjust the task order or path so that the robot can flexibly respond to unexpected variable situations while performing the task.

[0123] For example, the robot control system (1000) can assign a task in a complex environment to a specific robot that has excellent obstacle avoidance ability in a complex path, and conversely, can assign a relatively simple task to a robot that consumes energy quickly, thereby increasing the efficiency of the overall task.

[0124] As another example, the robot control system (1000) can reassign tasks to paths requiring charging when the robot's battery status unexpectedly deteriorates, or assign complex tasks to robots with superior obstacle avoidance capabilities, thereby enabling the robot to perform tasks more stably and efficiently.

[0125]

[0126] Next, the user interface provided to the user for simulation will be described with reference to FIGS. 7 and 8.

[0127] FIG. 7 and FIG. 8 are exemplary diagrams showing a user interface for simulation according to one embodiment of the present invention.

[0128] First, referring to FIG. 7, the robot control system (1000) can provide a user interface to the user through an interface module (1100) so that the user can check the simulation process and various data acquired through the simulation.

[0129] Specifically, a simulation situation can be displayed in real time in the first area (71) of the user interface. Here, the simulation situation can be a scene in which a robot operates in a virtual environment.

[0130] Additionally, the second area of ​​the user interface (73) may display the progress of the service scenario, and may display which task of the service scenario the currently displayed simulation situation is.

[0131] Additionally, the third area (75) of the user interface may display a robot status corresponding to the current simulation situation, and the fourth area (77) may display an event situation that occurred in the current simulation situation or an added event situation.

[0132] Next, referring to FIG. 8, the robot control system (1000) can provide a user interface to the user to set various conditions of the simulation through the interface module (1100).

[0133] Specifically, the user can input service scenarios to be used for simulation or change their order, etc. through the first area (81) of the user interface, and can set various conditions for the location or locations where the simulation will be performed through the second area (83).

[0134] Additionally, the user can select a robot and check or set the status of each robot through the third area (85) of the user interface, and input an event to be applied to the simulation through the fourth area (87).

[0135] That is, users can set conditions for the virtual environment through the user interface or check situations simulated in the virtual environment in real time.

[0136] Meanwhile, the user interface for the simulation of the above-described drawings 7 and 8 is an example, and the user interface may include various functions such as BGM, TTS, and view / player functions of the UI screen.

[0137] For example, the user interface may include functions to control appropriate background music (BGM) or voice output (TTS) during simulation, and to provide layout or configuration of UI screens according to user needs.

[0138] Above, according to the scenario verification method of the present invention, by creating a virtual environment reflecting an actual environment, it is possible to test and verify in advance the tasks to be performed by the robot.

[0139] In addition, the present invention can reduce memory usage and shorten the loading time of a simulation through a lazy loading technique that loads objects or data elements only when necessary in a virtual environment.

[0140] In addition, the present invention can control the complexity of the simulation by managing static and dynamic elements by layer in a virtual environment, and can make the entire simulation lighter by deactivating unnecessary elements.

[0141] Additionally, the present invention can help to configure an optimal scenario by distributing a service scenario to a robot or re-establishing a work plan based on the verification results.

[0142] In addition, the present invention can significantly improve the reliability and safety of service robots by testing and resolving problems expected in a real environment in advance in a virtual environment.

[0143] Meanwhile, the robot control system (1000) of the present invention or each module within the robot control system (1000) can be implemented as a computing device, and this will be described with reference to FIG. 9.

[0144] Figure 9 is an exemplary diagram showing the configuration of a computing device according to one embodiment of the present invention.

[0145] Referring to FIG. 9, in some embodiments of the present invention, the robot control system (1000) may be implemented in the form of a computing device.

[0146] At least one of each module constituting the robot control system (1000) is implemented on a general-purpose computing processor and thus may include a processor (1008), an input / output I / O (1002), a memory (1004), an interface (1006), and a bus (1014). The processor (1008), the input / output device (1002), the memory (1004), and / or the interface (1006) may be coupled to each other via the bus (1014). The bus (1014) corresponds to a path through which data is transferred.

[0147] Specifically, the processor (1008) may include at least one of a Central Processing Unit (CPU), a Micro Processor Unit (MPU), a Micro Controller Unit (MCU), a Graphic Processing Unit (GPU), a microprocessor, a digital signal processor, a microcontroller, an application processor (AP), and logic elements capable of performing functions similar thereto.

[0148] The input / output device (1002) may include at least one of a keypad, a keyboard, a touchscreen, and a display device. The memory (1004) may store data and / or programs.

[0149] The interface (1006) may perform a function of transmitting data to or receiving data from a communication network. The interface (1006) may be wired or wireless. For example, the interface (1006) may include an antenna or a wired / wireless transceiver. The memory (1004) may further include high-speed DRAM and / or SRAM, etc., as a volatile operating memory that enhances the operation of the processor (1008) while protecting personal information.

[0150] Additionally, the memory (1004) stores programming and data configurations that provide the functionality of some or all of the modules described herein. For example, it may include logic for performing selected aspects of the service scenario verification method through simulation of a virtual environment according to the present embodiment.

[0151] A program or application is loaded with a set of instructions including each operation of the present invention described above stored in the memory (1004) and the processor is enabled to perform each operation. Here, each operation may include an operation of creating a virtual environment corresponding to an actual environment in which a service scenario will be executed, an operation of simulating the operation of a robot based on the service scenario in the created virtual environment, an operation of verifying the service scenario using the result data of the simulation, an operation of creating learning data based on the behavioral data of the robot corresponding to variable situations, etc.

[0152] The various embodiments described herein may be implemented in a recording medium readable by a computer or similar device, for example, using software, hardware, or a combination thereof.

[0153] In terms of hardware implementation, the embodiments described herein can be implemented using at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors, controllers, micro-controllers, microprocessors, and other electrical units for performing functions. In some cases, the embodiments described herein can be implemented as a control module itself.

[0154] In a software implementation, the procedures and functions described herein, as well as other embodiments, may be implemented as separate software modules. Each of these software modules may perform one or more of the functions and operations described herein. The software code may be implemented as a software application written in a suitable programming language. The software code may be stored in a memory module and executed by a control module.

[0155] The above description is merely an example of the technical idea of ​​the present invention, and those skilled in the art will appreciate that various modifications, changes, and substitutions can be made without departing from the essential characteristics of the present invention.

[0156] Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate, rather than limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by these embodiments and the accompanying drawings. The protection scope of the present invention should be interpreted by the following claims, and all technical concepts within the scope equivalent thereto should be interpreted as being included within the scope of the rights of the present invention.

Claims

1. A method for verifying a service scenario through a virtual environment simulation performed on a computing device, A step of creating a virtual environment corresponding to the actual environment in which the service scenario will be executed; A step of simulating the robot's operation based on the above service scenario in a generated virtual environment; A service scenario verification method comprising a step of verifying the service scenario using the result data of the above simulation.

2. In paragraph 1, A service scenario verification method characterized in that the above simulating step is performed by adding an intended variable situation to the virtual environment.

3. In paragraph 2, A service scenario verification method, characterized in that the above variable situation includes at least one of a sensor malfunction of the robot, a change in battery status, and a network connection problem.

4. In paragraph 1, The steps to verify the above service scenario are A service scenario verification method characterized in that the service scenario is verified by evaluating whether the service scenario can achieve the expected target standard in an actual environment based on the above result data.

5. In paragraph 4, A service scenario verification method, characterized in that the above target criteria include the task success rate, time efficiency, energy consumption, and safety of the robot.

6. In paragraph 2, A service scenario verification method further comprising a step of generating learning data based on the robot's behavioral data corresponding to the above variable situation.

7. Processor, and including a memory communicating with the processor, The above memory stores instructions that cause the processor to perform operations, The above actions are actions for creating a virtual environment corresponding to the actual environment in which the service scenario will be executed; An action to simulate the robot's actions based on the above service scenario in a generated virtual environment; and A computing device comprising: an operation for verifying the service scenario using the result data of the above simulation; 8. In paragraph 7, A computing device characterized in that the above-mentioned simulating operation is simulated by adding an intended variable situation to the virtual environment.

9. In paragraph 8, A computing device characterized in that the above variable situation includes at least one of a sensor malfunction of the robot, a change in battery status, and a network connection problem.

10. In paragraph 7, A computing device characterized in that the operation of verifying the above service scenario verifies the service scenario by evaluating whether the service scenario can achieve the expected target standard in an actual environment based on the result data.

11. In paragraph 10, A computing device characterized in that the above target criteria include the task success rate, time efficiency, energy consumption and safety of the robot.

12. In paragraph 8, A computing device further comprising: an operation of generating learning data based on the robot's behavioral data corresponding to the above variable situation.

13. A computer-readable recording medium storing a program for performing a service scenario verification method according to any one of paragraphs 1 to 6.

14. A program stored on a computer-readable recording medium including a program code for executing a service scenario verification method according to any one of paragraphs 1 to 6.

Images