Comfortable user guidance by mobile companion robots

Abstract

To propose a computer implementation method and system for controlling a mobile robot accompanying at least one user. [Solution] The method includes obtaining information about at least one user and predicting the intent of movement of at least one user based on the information obtained about at least one user. The method includes obtaining information about the physiological state of at least one user, determining the user state of at least one user based on the information obtained about the physiological state, and determining at least one external factor that affects the determined user state. The method then plans the behavior of a mobile robot that maximizes a user comfort index determined based on the determined user state and the determined at least one external factor, generates control signals to control the mobile robot to perform the planned behavior, and controls the mobile robot based on the generated control signals.

Description

[Technical Field]

[0001] This invention relates to the field of mobile robots and behavioral planning for mobile robots. More specifically, a computer implementation method and system for controlling a mobile companion robot is proposed. [Background technology]

[0002] Robot companions accompany pedestrians, guiding them to their target locations. Mobile robots accompany users in shopping malls and museums, providing guidance as they move through the environment and assisting with navigation to specific target locations. Another application of mobile robots is to act as a partner during movement.

[0003] However, the behavioral planning for mobile robots that serve as companions to human users currently focuses on objectives such as guiding the user to their destination, preferably in a short amount of time. Factors such as the comfort of the accompanying user are not considered when planning the future behavior of the mobile robot.

[0004] Recent methods of navigation route planning take into account external factors, such as environmental factors (environmental parameters), when planning navigation routes.

[0005] For example, CN115855092A discloses a method for planning navigation routes for people, which aims to improve the human travel experience by planning navigation routes that take into account shaded areas on roads. The method calculates shaded or sunlit areas along a road based on the altitude and direction of the sun, as well as shaded objects such as buildings and trees along the road. The options for the planned navigation route for the user to reach their destination are based on the calculated shaded or sunlit areas, and may also take into account the season, such as summer or winter, or the ambient temperature during travel. The recommended navigation route options will thus increase the user's comfort while traveling along the selected navigation route, especially when walking or cycling, by having a higher shade coverage ratio in summer and a higher proportion of sunlit sections in winter.

[0006] This conventional technology plans navigation routes based on external factors, such as sunlight exposure, in addition to the primary objective of reaching a destination. Pedestrian users are presented with navigation route recommendations that consider only sunlight exposure as an environmental factor, but the user is left to decide which of the recommended options best suits their personal needs.

[0007] In another area of ​​focus, U.S. Patent Application Publication 2023 / 0052474(A1) relates to vehicle operation, correlating the physiological state of a vehicle occupant with the operating parameters of the vehicle. The system monitors the occupant's physiological parameters, detects the physiological state based on the monitored physiological parameters, determines the correlation between the vehicle's operating parameters and the detected physiological state, and adapts the vehicle's operating parameters to address that physiological state. By causing a change in the vehicle's autonomous operation (which may include changing the navigation route to the destination), the system attempts to avoid negative impacts on the occupant's physiological state. [Overview of the project] [Problems that the invention aims to solve]

[0008] There remains a problem of improving the current method of behavior planning of a mobile robot that accompanies a user to improve the user's comfort, which is not considered in the current method of navigation planning of autonomous agents. **Means for Solving the Problem**

[0009] The computer-implemented method of claim 1, and a system for controlling the movement of a mobile robot according to the independent claim of the corresponding system, provide a convenient solution to this problem.

[0010] The dependent claims define further advantageous embodiments.

[0011] A computer-implemented method for controlling a mobile robot that accompanies at least one user includes obtaining information about at least one user, predicting the intention of the user's movement based on the obtained information about past movements, obtaining information about the physiological state of at least one user, determining the user state of at least one user based on the obtained information about the physiological state, and determining at least one external factor that affects the determined user state. The method further includes planning the behavior of the mobile robot to maximize a user comfort index determined based on the determined user state and the determined at least one external factor, generating a control signal for controlling the mobile robot to perform the planned behavior, and controlling the mobile robot based on the generated control signal.

[0012] The mobile robot is a mobile companion robot, for example, an autonomously operating mobile device that accompanies a user or a group of users moving within an environment. The mobile robot or robot companion may function as a museum guide robot, a shopping companion robot, or an exercise partner robot for at least one user when performing physical exercise.

[0013] The mobile robot may be one of a micromobility robot, a wheeled robot, a legged robot, a rail-mounted robot, or an aerial drone or an underwater drone.

[0014] At least one external factor defines characteristics in the environment that affect the user state of a human user, the user states of multiple users, or the user states of a group of users. At least one external factor may represent characteristics of the environment, such as ambient temperature, or specific elements in the environment, such as the number of curves and the radius of the curves of a user's movement path along a road or a sidewalk.

[0015] Information about at least one user may include information about the past movements of at least one user. Movement occurs when an object, in the context of this specification a user, changes its position relative to a reference point as time passes. Movement may be described in terms of displacement, distance, direction, speed, velocity, acceleration, deceleration, rotation, and a reference system.

[0016] Past movements are the user's movements from immediately before the current time to the current time. The intention of the user's movement includes, for example, the intended route (movement path) of the user.

[0017] At least one user is a human user, particularly a walking human user.

[0018] The user state (the human state of the user) includes, for example, the emotional state of a human.

[0019] <00001I1>The planned behavior includes the planned route (movement path) of the mobile robot in the environment.

[0020] The computer implementation method provides a framework for planning the behavior of a mobile robot accompanying a user, by determining external factors that influence the user state, such as the emotional and physical state of a user accompanied by a mobile robot, and using these factors in the behavior planning process. The method allows monitoring and inferring the user's user state, predicting whether the user's intended movements will affect the monitored user state, and planning the behavior of the mobile robot that positively influences the user's user state.

[0021] In one embodiment of the computer implementation method, information regarding the physiological state includes information regarding at least one of the user's pulse rate, heart rate, blood oxygen saturation, blood pressure, and body temperature.

[0022] Specific information regarding physiological states can be measured, for example, by widely used wearable sensors, which makes it possible to determine the user state of a human user.

[0023] One embodiment of the computer implementation method includes determining the user state based on image information of the user's face or image information of the user's body shape.

[0024] Current mobile robots are generally equipped with camera sensors, for example, for navigation purposes. Processing image information representing the user using known processing techniques makes it possible to determine the user's user state and thus obtain the information that serves as a basis for adapting the mobile robot's behavior plan to take the user state into consideration.

[0025] A user's body shape can significantly influence which of their actions, guided by a mobile robot within an environment, affect their comfort during the execution of those actions. Therefore, planning robot behavior while taking the user's body shape into consideration has a significant impact on the user's comfort and user state when being guided by a mobile robot.

[0026] A computer implementation method according to one embodiment includes determining the user state based on information about the user's physiological state, including sensor information acquired from at least one sensor of a mobile robot, particularly from a camera sensor that captures image information representing the user.

[0027] Using camera sensors on mobile robots efficiently utilizes already available sensors when implementing this method. Therefore, implementation costs are not increased by additional, expensive (camera) hardware.

[0028] At least one sensor may include a wearable sensor worn by the user, such as a smartwatch worn by the user.

[0029] Wearable sensors are becoming increasingly widespread and provide a wide range of measurements that can be conveniently used to determine a user's state. Integrating wearable sensors with wireless communication interfaces into systems implementing this method is generally cost-effective because it eliminates the need to design and add proprietary sensors, which would incur additional hardware costs.

[0030] According to one embodiment of the computer implementation method, at least one external factor includes crowd density, walking time, walking distance, spatial density of the user group, ambient temperature, shaded areas in the environment, sunlit areas in the environment, speed, and the number and arrangement of curves along the travel path.

[0031] The cited examples of external factors are currently the most overlooked in the behavior planning of mobile robots that accompany users, but they can significantly impact the user's comfort experience when being guided by a mobile robot. Taking these external factors into consideration during the behavior planning process would increase the consideration and thoughtfulness towards the user demonstrated by the mobile robot's behavior, and thus increase user acceptance of being accompanied by a mobile robot.

[0032] In one embodiment of the computer implementation method, determining external factors includes at least one of the following: sensing rainfall or snowfall with a membrane capacity rain sensor or camera sensor; sensing ambient temperature with a temperature sensor; determining sunlight intensity, particularly the spatial distribution of sunlight intensity in the environment of the mobile robot, with a camera sensor; sensing dazzling light with a light sensor; sensing at least one of wind direction and wind strength with a wind sensor; capturing images of surfaces with a camera sensor; and performing image processing on the captured images with a processor to determine at least one of a rocky surface, a rough surface, a wet surface, or a dirty road surface, or a road gradient; obtaining at least one of traffic congestion information, the number of visitors to a store or museum, or waiting times from a database stored on a server; and obtaining sensor information for determining at least one external factor with the communication unit of the mobile robot.

[0033] The cited external factors are easily measurable and quantifiable, providing factors, variables, or parameters that significantly influence the quality of the planned behavior when considered in the behavior planning process of a mobile robot.

[0034] One embodiment of a computer implementation method includes predicting the user's intent to move. For example, this includes predicting the user's path using a constant-velocity model based on information obtained from at least one user.

[0035] The use of a constant-velocity model provides a computationally efficient method for predicting user intent and enables cost-effective implementation of this method when used to predict user intent.

[0036] In one embodiment of the computer implementation method, planning the behavior of a mobile robot includes planning a mobile robot path that guides the user along a different path from the intended path of the user's anticipated movement intention if the user comfort index of the planned path exceeds the user comfort index of the intended movement path of the user's movement intention, or planning a mobile robot path that guides the user along the intended movement path of the user's anticipated movement intention if the user comfort index of the planned path does not exceed the comfort index of the intended movement path of the user's anticipated movement intention.

[0037] Therefore, the method generally works by planning and controlling the behavior of a mobile robot that follows the user's intentions, and by guiding the user through the behavior of the mobile robot when it is determined that the user cannot maintain comfort through their own intentions of movement. Thus, it can be expected that the robot companion controlled by this computer implementation method will be well-received by users.

[0038] In one embodiment of the computer implementation method, planning the behavior of a mobile robot includes determining an discomfort index based on a determined user state and at least one external factor, and, if the determined index exceeds a threshold, adjusting parameters (variables, factors) in the behavior planner according to the determined user state, the user's expected movement intention, and at least one external factor.

[0039] Therefore, the method plans the behavior of a mobile robot to guide the user through actions that are only performed when it is determined that the user cannot maintain comfort through their own intention to move.

[0040] In one embodiment of the computer implementation method, fitting the parameters of the behavior planner includes at least one of fitting the cost components of the total cost function used in the behavior planner, adding behavior options for the mobile robot, and fitting the weights of the cost components used in the behavior planner.

[0041] Adapting the behavior planner parameters according to this computer implementation method makes it possible to incorporate this advantageous technique into the current behavior planning and control structures of mobile companion robots at limited cost.

[0042] According to one embodiment of the computer implementation method, planning the behavior of a mobile robot can be done using a cost-based planner, particularly a collaborative risk map planner, or a search-based planner, for example, A * Alternatively, this may include planning behavior using a planner based on Monte Carlo tree search, a sampling-based planner such as one based on RRT or a probabilistic roadmap, or a reinforcement learning planner such as one based on Q-learning or a PPO.

[0043] The computer implementation method can be carried out using a wide range of cost-based behavioral planning algorithms with only minor modifications by adapting the parameters of the behavioral planning algorithm used, and therefore the implementation of this method is cost-effective. For example, the optimization planning algorithms disclosed in U.S. Patent Publication No. 11,220,261(B2) and U.S. Patent No. 11,679,782(B2) provide a favorable basis for the implementation of this computer implementation method.

[0044] In one embodiment of the computer implementation method, controlling a mobile robot based on generated control signals includes at least one of the following: controlling the movement of the mobile robot by a controller, and generating output information based on the planned behavior, and outputting the generated output information through the mobile robot's human-machine interface.

[0045] The human-machine interface may include at least one of the following: voice-based assistance to the user by the mobile robot, and visual output via a display device or projector of the mobile robot.

[0046] The human-machine interface may provide the user with recommendations for multiple behaviors, via visual or audio output, to select one of those recommendations for the mobile robot to perform.

[0047] Therefore, the computer implementation method effectively utilizes current modalities for providing guidance by mobile robots to human users accompanied by them.

[0048] A system for controlling a mobile robot configured to accompany at least one user includes acquisition means for acquiring information about at least one user. The system further includes prediction means for predicting the user's movement intentions based on acquired information about at least one user, and user information acquisition means for acquiring information about the user's physiological state. The system's user state determination means is configured to determine the user state based on acquired information about the physiological state. The system further includes factor determination means for determining at least one external factor that affects the determined user state, and a behavior planner for planning the behavior of the mobile robot to maximize a user comfort index determined based on the determined user state and the determined at least one external factor. The system includes at least one of a controller for generating control signals to implement the planned behavior and controlling the mobile robot's actuators based on the generated control signals, and a human-machine interface for generating output information based on the mobile robot's planned behavior and outputting the generated output information to the user.

[0049] The controller controls the mobile robot's actuators to perform actions that indirectly improve user comfort, for example, by slowing down the mobile robot's movement to allow the user to approach. Additionally or alternatively, the controller controls the mobile robot's actuators to perform actions that directly improve user comfort, such as by guiding the mobile robot to move along a planned path to avoid obstacles or along a path that is not too steep.

[0050] The following description of the embodiments will be illustrated with reference to the following figure. [Brief explanation of the drawing]

[0051] [Figure 1] This is a schematic flowchart of a computer implementation method according to one embodiment. [Figure 2] This is a simplified block diagram of the system in one embodiment. [Figure 3] This is an explanatory diagram illustrating application scenarios for computer implementation methods. [Figure 4] This is an explanatory diagram illustrating another application scenario for computer implementation methods. [Figure 5] This diagram illustrates crowd density as an external factor in one application example. [Figure 6] This diagram illustrates the division of user groups as an external factor in one application example. [Figure 7] This diagram illustrates walking time as an external factor in one application example. [Figure 8] This diagram illustrates the processing of the behavior planner in the first application example. [Figure 9] This diagram illustrates the processing of the behavior planner in the second application example. [Figure 10] This is a high-level abstraction diagram of a hardware architecture suitable for system implementation. [Modes for carrying out the invention]

[0052] In the diagrams, corresponding elements share the same reference numeral. In the diagram descriptions, explanations of the same reference numerals in different diagrams should be avoided where possible, without negatively impacting comprehension, and unnecessary repetition should be avoided for the sake of brevity.

[0053] Figure 1 shows a schematic flowchart of a computer implementation method according to one embodiment.

[0054] The following embodiments will be described with reference to at least one user 20, which is one user accompanying the mobile robot 1, unless otherwise explicitly stated.

[0055] The information relating to at least one user 20 in the following description of the embodiments is information relating to the past movements of at least one user 20.

[0056] The method begins with step S1, which includes obtaining information about the past movements of at least one user 20.

[0057] In step S2, the method then predicts the user's intent 12 to move based on the acquired information about the past movements of at least one user 20.

[0058] In step S3, the method then obtains information about the physiological state of user 20. In step S4, the method determines the user state 13 (user state) of user 20 based on the obtained information about the physiological state.

[0059] In step S9, the method compares the determined user state of user 20 with the user state threshold. If the determined user state exceeds the user state threshold, the method proceeds to step S6.

[0060] If the determined user state does not exceed the user state threshold, the method proceeds to step S8.

[0061] In step S6, the method determines at least one external factor that affects the determined user state.

[0062] External factors or external parameters refer to external variables that can change over time and whose actual values ​​affect the determination of user states, particularly which user states are actually determined.

[0063] In step S7, which the method performs after step S6, the method plans the behavior 16 of the mobile robot 1 that maximizes a user comfort index determined based on the user state 13 determined in step S4 and at least one external factor determined in step S6.

[0064] In step S9, the method then performs the planned behavior 16.

[0065] In step S8, after determining that the determined user state 13 does not exceed the user state threshold, the method executes a conventional behavior plan to determine the behavior of the mobile robot in order to reach the target. After determining the behavior of the mobile robot 1 using the conventional behavior plan (conventional behavior) in step S8, the method proceeds from step S8 to step S9 and executes the planned behavior 16, in this case the planned conventional behavior.

[0066] Figure 2 shows a simplified block diagram illustrating the elements of system 100 in one embodiment.

[0067] System 100 is configured to control a mobile robot 1 that accompanies a human user 20 as it moves around the environment. Figure 2 specifically shows functional elements suitable for implementation as software modules, and further aspects of the hardware implementation of the system are described with reference to Figure 10.

[0068] The system 100 for controlling the mobile robot 1 accompanying the user 20 includes acquisition means for acquiring information about the user 20's past movements, which are not explicitly shown in Figure 2. The prediction means 2 of the system 100 predicts the user 20's intended movement 12 based on the acquired information about the user's past movements that the acquisition means provides to the prediction means 2.

[0069] The user's predicted movement intent 12 may include, for example, a predicted user path, which is predicted by the prediction means 2 using a constant-velocity model based on sensor information contained in information about the user's past movements 20.

[0070] As an addition or alternative, the prediction means 2 of the system 100 may predict the user 20's intended movement 12 based on acquired information, including information regarding the user's head angle and the user 20's gaze direction, which the acquisition means may acquire and evaluate and then provide to the prediction means 2.

[0071] The acquisition means may include acquiring sensor information from at least one external sensor 49 via the communication unit 41 of the mobile robot 1 in order to predict the user 20's intended movement 12 using the prediction means 2, thereby acquiring information about the user 20's past movements.

[0072] System 100 further includes user information acquisition means, which are not explicitly shown in Figure 2, for acquiring information about the physiological state of user 20. User state determination means 3 determines the user state 13 based on the acquired information about the user's physiological state, which is acquired by the user information acquisition means and provided to user state determination means 3 by the user information acquisition means.

[0073] The user information acquisition means may acquire information about the user's physiological state from at least one wearable sensor 49 of the user. For example, the user information acquisition means may acquire user information about the user's physical state via the user's smartphone, or directly from or indirectly via the user's smartphone from a smartwatch worn by the user.

[0074] A smartwatch may measure the user's pulse rate, and a user information acquisition means may acquire information about the user's physiological state as pulse rate information from the smartwatch.

[0075] Alternatively or additionally, the user information acquisition means may acquire information about the user's physiological state from one or more sensors 45 of the mobile robot 1. The mobile robot 1 includes a camera sensor mounted on the robot's main frame. The camera sensor captures an image or a sequence of images (video) of the user, particularly representing the user's face. By performing appropriate image processing on the captured images of the user's facial expressions included in the information about the user's physiological state, the user state determination means 3 determines the user's user state 13 based on the acquired information about the user's physiological state, including information about the user's facial expressions. Each facial expression of the user serves as an indicator of the current user state 13, specifically the user's current emotional state, including, for example, an angry user state, a sad user state, or a happy user state.

[0076] Alternatively or additionally, the user information acquisition means may acquire information about the user's physiological state, including image information of the user's body, as camera information from a camera sensor, for example, a camera sensor attached to the main frame of the mobile robot 1. The user state determination means 3 then determines the user state 13, which includes information about the user's body shape. The information about the user's body shape provided to the improved behavior planner 19 enables the planning of the behavior of the mobile robot 1, including appropriate velocity values ​​and movement path curve characteristics that are considered favorable to the user, as described below.

[0077] The user state 13 determined by the user state determination means 3 may include information regarding at least one of the user's pulse rate, heart rate, blood oxygen saturation, blood pressure, or body temperature.

[0078] The user information acquisition means 6 may acquire information about the user's physiological state, including acquiring sensor status from at least one external sensor 49 via the communication unit 41 of the mobile robot 1, in order to determine the user state 13 by the user state determination means 3.

[0079] The system 100 further includes an external factor determination means 6 (determination means 6) that determines at least one external parameter 14 that affects the determined user state 13. The factor determination means 6 may obtain at least one external factor 14 from a database 7 (first database 7) that stores a plurality of predetermined external factors 14 related to a particular behavior planner and a particular environment.

[0080] Alternatively, or in addition, the factor determination means 6 may determine at least one external factor, including information regarding the density of people in the user's environment.

[0081] At least one external factor may include information about the user's walking time to reach a specific goal within the environment.

[0082] At least one external factor may include information about the walking distance to a specific target location within the environment.

[0083] At least one external factor may include information about the spatial density of a group of users, including multiple users within the environment. A group of users could be, for example, a group of travelers or visitors to a museum, tourist attraction, or shopping mall.

[0084] At least one external factor may include information regarding ambient temperature within the user's environment or along the planned travel path (travel path options) of the mobile robot 1 accompanied by the user.

[0085] At least one external factor may include information about at least one shaded area in the environment, or at least one sunny area in the environment. The factor determination means 6 determines information about at least one shaded area or at least one sunny area in the environment, or along the movement path options of the mobile robot 1, based on weather data including season, time, current weather data and weather forecast data, and information about physical objects including the size and location of the objects.

[0086] At least one external factor may include information regarding speed, the number and arrangement of curves along the chosen travel path.

[0087] The factor determination means 6 may acquire spatial information regarding movement path options from the core behavior planner 9 of the improved behavior planner 19, as shown in Figure 2.

[0088] The factor determination means 6 may determine external parameters by sensing rainfall or snowfall, for example, using a membrane capacity rain sensor or a camera sensor.

[0089] The factor determination means 6 may determine external factors, for example, by sensing the ambient temperature in the environment using a temperature sensor.

[0090] The factor determination means 6 may determine external factors, for example, by using a camera sensor to measure the spatial distribution of sunlight intensity, particularly the sunlight intensity within the environment of the mobile robot 1.

[0091] The factor determination means 6 may determine external factors, for example, by detecting dazzling light using a light sensor. Dazzling light is light with an incident direction and light intensity (brightness) that could blind the user 20 when following the movement path of the mobile robot 1.

[0092] The factor determination means 6 may determine external factors by, for example, using a wind sensor to sense at least one of the wind direction and wind strength in the environment or along the planned movement path options of the mobile robot 1.

[0093] The factor determination means 6 may determine external factors by, for example, using a camera sensor to capture an image of the surface and performing image processing on the captured image using a processor to determine at least one of the following: a rocky surface, a rough surface, a wet surface, a dirty road surface, or the gradient of the road.

[0094] The factor determination means 6 may determine external parameters by, for example, obtaining at least one of the following from the database 7 stored in the server 50: traffic congestion information, stores, museums, shopping malls, the number of visitors to stores, and waiting times for stores, museums, shopping malls, stores, or tourist destinations.

[0095] The factor determination means 6 may determine at least one external factor by, for example, using the communication unit 41 of the mobile robot 1 to acquire sensor information from at least one external sensor 49.

[0096] System 100 includes a behavior planner 19 (improved behavior planner 19) comprising a behavior planner adaptation means 8 and a core behavior planner 9.

[0097] System 100 further includes database 10 (second database 10). Alternatively or additionally, behavior planner adaptation means 8 has access to a database outside of behavior planner 19, which corresponds to database 10.

[0098] The improved behavior planner 9 plans the behavior 16 of the mobile robot 100 that maximizes a user comfort index, which is determined based on the determined user state 13, the user's 20's predicted movement intention 12, and at least one determined external factor 14.

[0099] Behavior 16 may include a behavior plan, which includes at least one of a planned movement path (planned route) of the mobile robot 1 and a communication plan that includes the communication of information to the user 20 via the human-machine interface 5.

[0100] In detail, the behavior planner adaptation means 8 adapts the components of the behavior planner 9 according to the user state 13, the user's intent to move 20, and external factors 14.

[0101] The behavior planner adaptation means 8 detects user 20 discomfort based on user state 13. In a specific application example, the behavior planner adaptation means 8 determines, based on user state 13, that user 20's heart rate exceeds a heart rate threshold, and further determines that external factors 14 include a high ambient temperature, and detects the presence of a shaded area along the direction of movement corresponding to user 20's intended movement 12.

[0102] The behavior planner adaptation means 8 adds, for example, a comfort cost component for traversing shaded areas, based on thresholds and rules obtained from the database 7.

[0103] The behavior planner fitting means 8 determines whether the predicted movement intention 12 alone improves the user 20's comfort, and if it determines that the movement intention 12 cannot improve the comfort index determined for the user 20, it may provide the core behavior planner 9 with additional behavioral options for the mobile robot 1. The core behavior planner may then plan the behavior of the mobile robot 1 to guide the user 20 through the environment along shaded areas in the direction of the user 20's predicted movement intention.

[0104] The behavior planner fitting means 8 may calculate additional comfort cost component weights, such as shade comfort weights, based on the user state 13 or based on the tracked interaction (negotiation) intensity between the comfortable guidance actions performed by the mobile robot 1 and the corresponding responses of the user 20.

[0105] The behavior planner fitting means 8 may use a state machine or a behavior tree to fit the parameters of the core behavior planner 9. To give a concrete example, the first task may correspond to the user 20 reaching a specific goal, and the second task, based on the switch setting, may represent addressing the first task again, which is to improve the user state 13 by going to a shaded area and then reach the goal.

[0106] Alternatively, the behavior planner fitting means 8 obtains the relationship between the user state 13, the external factors 14, and the proposed fitting of corresponding parameters to the core behavior planner 9 by querying a large-scale language model.

[0107] The behavior planner adaptation means 8 provides the determined cost components, behaviors, and weights to the core behavior planner 9 in order to plan the behavior 16 of the mobile robot 1. In an example where the user 20 is guided toward a target in a high-temperature environment exceeding the user 20's comfort temperature level, the behavior planner adaptation means 8 provides the core behavior planner 9 with the cost component of shade comfort benefit, the cost component of the user's target speed, the behavior option of "wait in a shaded area" and the behavior option of "take a break at a coffee shop".

[0108] Based on the determined cost components, behavior, and weights provided to the core behavior planner 9 as the adapted planning parameters 15, the core behavior planner 9 plans the behavior 16 of the mobile robot 1.

[0109] In this particular example, the behavior planner 9 plans the behavior of the mobile robot 1, including a movement path through shaded areas in the environment, while simultaneously ensuring that it avoids static and dynamic objects in the environment and reaches the user's objective as indicated in the predicted movement intent 12. To plan the behavior of the mobile robot 1, the core behavior planner 9 further uses sensor information about the environment of the mobile robot 1, obtained via the mobile robot 1's sensors 45. Alternatively or additionally, the mobile robot 1 also uses sensor information about the environment obtained via the communication interface 41 from sensors 49 located outside the mobile robot 1.

[0110] The core behavior planner 9 is implemented as a cost-based planner for planning the behavior of autonomously operating entities. For example, the behavior planner 9 may be implemented using a collaborative risk map planner, such as those described in detail in U.S. Patent Publications 11,220,261(B2) and 11,679,782(B2).

[0111] Alternative implementations of Core Behavior Planner 9 include different cost-based planning algorithms, such as search-based planners, e.g., A * Alternatively, a planner based on Monte Carlo tree search, a sampling-based planner such as one based on RRT or a probabilistic roadmap, or a reinforcement learning planner such as one based on Q-learning or a PPO-based planner can be used.

[0112] The system 100 further includes at least one of a controller 4 and a human-machine interface 5.

[0113] The controller 4 generates a control signal 17 to execute the planned behavior 16 provided by the behavior planner 9. The controller 4 outputs the generated control signal 17 to the actuator 48 of the mobile robot 1 and controls the actuator 48 of the mobile robot 1 via the generated control signal 17.

[0114] The actuator 48 enables the planned movement of the mobile robot 1 based on the control signal 17. For example, the actuator 4 includes the legs of the humanoid mobile robot 1, which are actuated based on the control signal 17 to move the mobile robot 1 along a planned movement path in the environment, for example, to guide the user 20 along the planned path of the mobile robot 1. In a particular example, the system 100 controls the actuator 4 of the mobile robot 1 to brake the mobile robot 1 and reduce its speed so that the user 20 can approach the mobile robot 1, or, around obstacles, to change the direction of movement of the moving mobile robot 1 to guide the user 20 along a movement path with a conveniently gentle slope.

[0115] The human-machine interface 5 generates an output signal 18 containing output information based on the planned behavior 16 of the mobile robot 1 provided by the improved behavior planner 9, and outputs the output signal to an output means, such as a display, one or more loudspeakers, or headphones worn by the user 20. The output means communicates the output information to the user 20 via an acoustic signal, a visual signal, a tactile signal, or a combination thereof.

[0116] The human-machine interface 5 may output information to the user, including recommendations for behavioral options for the mobile robot 1 and the user 20 accompanying the mobile robot 1. In a particular example, the system 100 controls the user interface 5 of the mobile robot 1 to provide voice assistance to the user 20 by outputting, for example, via a loudspeaker, "Look to the right here. Wouldn't you like to walk in the shade?" in order to guide the user 20 to walk in the shade, thereby improving the user 20's comfort in light of the environment characterized by high temperatures and the user 20's sensitivity to high temperatures.

[0117] In the same specific example, system 100 controls the user interface 5 of mobile robot 1 to provide visual assistance to user 20 by, for example, outputting a message via the display saying "Shade is better" to guide user 20 to walk in the shade, thereby improving user 20's comfort in an environment characterized by high temperature and user 20's sensitivity to high temperature.

[0118] As an addition or alternative, the human-machine interface 5 may visually or audibly output information to the user 20, including multiple different recommendations for behavioral options for the mobile robot 1 and the user 20 accompanying the mobile robot 1. In the same particular example, the system 100 controls the user interface 5 of the mobile robot 1 to provide visual assistance to the user 20 by outputting information to the user 20, including alternative options, such as a message saying, "You can wait in the shade near the building, or you can slow down your walking speed and continue."

[0119] Figure 3 illustrates an application scenario for the computer implementation method. This application scenario includes a mobile robot 1 that functions as a companion robot accompanying a human user 20 as they walk through an environment. Specifically, the mobile robot 1 guides the user 20, which essentially means that the user 20 follows the path that the mobile robot 1 travels through the environment. Thus, the mobile robot 1 guides the user 20.

[0120] The system 100 according to the embodiment described with reference to Figures 1 and 2 plans and controls the behavior performed by the mobile robot 1. The system 100 determines the user state 13 of the user 20 by determining external factors 14 and influences the user state 13. The system 100 determines whether the user 20's predicted movement intentions improve the user 20's determined comfort index. If the user 20's predicted movement intentions have the ability to improve the user 20's determined comfort index, the system 100 plans the behavior of the mobile robot 1 in a known conventional manner, and in detail without adapting the parameters of the behavior planner 19 by the behavior planner adaptation means 8.

[0121] The top of Figure 3 shows the user state 13 over time. At the current time t, the user state 13, including, for example, the user 20's pulse rate, has already exceeded a threshold for some time. The system 100 predicts the user 20's intended movements 12 that will improve user comfort, as shown by the continuation curve 22 of the user state 13 for time after time t.

[0122] Alternatively, system 100 predicts user 20 movement intentions 12 that do not improve user comfort, as shown by the continuation curve 21 of user state 13 for time after time t.

[0123] If the predicted movement intentions of user 20 cannot improve the comfort index determined by user 20, system 100 plans the behavior of mobile robot 1 according to the procedure shown in the central part of Figure 3.

[0124] The system 100, in particular the behavior planner fitting means 8, then fits the parameters of the behavior planner 19.

[0125] For example, according to Figure 3, the cost of the system 100 is the cost of the comfortable target speed c. speed Add this to the cost of behavior planner 19.

[0126] Furthermore, System 100 has a cost of curve comfort C curve Add this to the cost of behavior planner 19.

[0127] The behavior planner 19 plans the behavior of the mobile robot 1 using the adapted cost in the cost-based behavior planning process of the core behavior planner 9. Cost C speed Due to the adapted costs, including cost component C, the planned behavior of mobile robot 1 does not include the adapted costs, and therefore cost component C. speed The mobile robot 1 has a reduced speed when compared to the results of the behavior planning process, which does not consider the adapted costs, including the reduced speed. This is shown as a sub-figure A in the central part of Figure 3.

[0128] Cost C curve Due to the adapted costs, including cost component C, the planned behavior of mobile robot 1 does not include the adapted costs, and therefore cost component C. curve This includes a straightened movement path for mobile robot 1 with fewer curves, compared to the results of a behavior planning process that does not consider adapted costs. This is shown as a sub-figure B in the central part of Figure 3.

[0129] As shown in the lower part of Figure 3, the improved behavior planning process performed by System 100 enables System 100 to control the movement of the mobile robot 1 so as to induce user comfort for User 20, specifically by reducing User 20's pulse rate to a low value, particularly to the low pulse rate value shown by curve 22, as User 20 follows the movement of the mobile robot 1. The value of pulse rate curve 22 is below the pulse rate threshold level and is also smaller than the value of pulse rate curve 21 when the behavior planner 19 does not fit the cost of behavior planning.

[0130] In contrast to the processing in System 100 and the computer implementation method in Figure 1, known methods for planning the behavior of an autonomous device acting as a robot companion do not consider the user state 13 of the user 20. Instead, System 100 considers the intention of human movement 12, along with external factors 14 that affect the comfort of the user 20 as defined by a user comfort index, for the behavior planning process of the mobile robot 1.

[0131] Figure 4 shows another application scenario for the computer implementation method. This application scenario involves a user 20 walking with a mobile robot 1 controlled by system 100 in an environment with ambient temperatures exceeding the generally acceptable temperature range for a human user, for example, the ambient temperature exceeding a temperature range of 25°C or 30°C.

[0132] Ambient temperature represents an example of an external factor that affects the comfort of user 20 while walking along a movement path in the environment, guided by mobile robot 1. Furthermore, the environment includes shaded areas, and therefore areas where sunlight exposure is reduced compared to areas where mobile robot 1 and user 20 accompanying mobile robot 1 are exposed to direct sunlight.

[0133] System 100 determines the user state 13 of user 20 and senses a high heart rate of user 20, for example, a heart rate that exceeds the human comfort level and thereby indicates discomfort for user 20 as user state 13. System 100 associates the determined uncomfortable user state 13, including the high heart rate, with the high temperature measured in the environment.

[0134] System 100 determines the ambient temperature, which is an external factor, and detects shaded areas in the environment that extend near the intended travel path 23 of the user 20 accompanying the mobile robot 1.

[0135] When the system 100 determines that the user 20 improves the comfort index by executing the movement intention 12 and moving along the movement path 23 on the lower path shown in the figure, the system 100 performs the behavior planning without adapting the parameters of the behavior planning process in the behavior planner 19. The mobile robot moves along the movement path 23 as shown in the lower part of FIG. 4.

[0136] Human comfort index, specifically the benefit C of the shaded comfort of the user 20 following the mobile robot 1 shadow depends on the time when the mobile robot travels in the shade and is shielded from direct sunlight irradiation. Then, the total cost to be considered by the improved behavior planner 19 is, for example, the collision risk R regarding collisions with static and dynamic objects in the environment collision , the effectiveness U of reaching the goal regarding the movement intention of the user 20 goal , and the benefit C of shaded comfort determined by the behavior planner adaptation means 8 shadow are included. Each cost component R collision , U goal , and C shadow is taken into account together with the corresponding cost weight w collision , w goal and w shadow and as a result, the following total post - adaptation cost function C total is obtained. C total = w collision R collision - w goal U goal - w shadow C shadow (1)

[0137] The system 100 performs the behavior planning of the mobile robot 1 based on the cost function according to formula (1). As a result, in order to follow the mobile robot 1 and approach the goal of the movement intention 12, a behavior including a movement path 24 that makes use of the benefit of the user 20 moving along the shaded area is obtained.

[0138] The weight w of shaded comfort shadowBecause the behavior planner adaptation means 8 is continuously modified according to the user 20's intended movements, the system 100 benefits the comfort of shade C. shadow The mobile robot 100 is controlled as it moves along the movement path 24, while continuously changing the cost components of the planner. The improved behavior planner 19 plans the behavior of the mobile robot 1 along the shaded areas from 1) when it enters the first shaded area to 3) when it leaves the third shaded area, as it moves along the target in the environment shown at the bottom of Figure 4.

[0139] System 100 may plan additional behavior of the mobile robot 1 in accordance with the determined human intention 12. If the determined human intention 12 allows, System 100 decides to add an additional waiting behavior of the mobile robot 1 in the first shaded area at time 2), thereby achieving a more favorable reduction in the pulse rate of the user state 13 while remaining stationary in the shaded area as shown at the top of Figure 4.

[0140] Generally, the planned movement path of the mobile robot 1 follows the determined movement intent 12 of the user 20. The prediction means 2 may predict the user 20's movement intent 12 using a constant speed model. Total cost C total Effective benefits included U goal This ensures that the mobile robot 1 can reach the user's target (target point) of the user's predicted movement intention 12 by following the movement path of the planned behavior 16. If the predicted movement intention 12 cannot improve the user comfort index on its own, the system 100 plans the behavior of the mobile robot 1 using an improved behavior planner 19, with additional cost components, including, for example, an additional cost component related to shade comfort benefits.

[0141] The behavior planner adaptation means 8 adjusts the required comfort C required by the user 20 according to formula (2). target And, estimated user comfort C user The weight of the shade comfort benefit in the cost component is determined, depending on the difference between the two. w shadow ~ΔC=Ctarget -C user (2)

[0142] Required C target This is determined by the current user state 13, for example, by the pulse rate. Estimated user comfort C user This is estimated based on the determined intent of the user's current actions.

[0143] Figure 5 shows crowd density as an external factor in one application example.

[0144] The scenario in Figure 5 concerns a user 20 accompanied by a mobile robot 1. System 100 predicts that the user 20's intended movement 12 includes an intended movement path 25. The intended movement path 25 passes near a group of other pedestrians 26.

[0145] In the example in Figure 5, the external factor 14 is crowd density. User 20's stress level depends on the distance to others. The stress level increases as the distance to other pedestrians 26 decreases.

[0146] The system 100 determines a comfort index for user 20 based on the estimated stress level of user 20, and the estimated stress level may be determined by the system 100 using sensors 49 attached to user 20, to give a few examples, such as a heart rate sensor, a blood oxygen sensor, a blood pressure sensor, a body temperature sensor.

[0147] System 100 determines high crowd density as an external factor 14 that affects the user state 13 in the environment, for example, within a pedestrian area where the predicted movement path 25 passes a short distance away. System 100 may determine crowd density based on signals from the camera sensor of a mobile robot 1 accompanying the user 20.

[0148] If system 100 determines a user state 13 that includes a high stress level for user 20 and predicts that the movement path 25 will not be able to maintain a sufficient distance from the group of other pedestrians 26, system 100 adapts the total cost function of behavior planner 19 to plan the behavior 16 of the mobile robot by adding an additional cost component (peripersonal cost component) to the cost. The behavior planner 19 then plans the behavior of the mobile robot 1, including a planned movement path 24 that results in an increased distance to the group of other pedestrians 26 compared to the predicted intended movement path 25. System 100, which controls the movement of the mobile robot 1 along the planned movement path 24, guides user 20 to a behavior that ultimately improves the user state 13 by avoiding the increased stress level caused by the short distance to the other pedestrians 26.

[0149] Figure 6 shows the external factor "Splitting of User Groups" in one application scenario.

[0150] External factors 14 that affect the user state 13 of each user 20 within the group of users 20 may include possible divisions of the group of users 20. One characteristic of the group of users is that users 20 who are part of the group will try to stay together when following the mobile robot 1 as a group of users 20. A scenario in which this behavior of users 20 can be used in the behavior planning of the mobile robot 1 by the system 100 is a mobile guide robot guiding a group of travelers including users 1, 2, and 3 in the scenario of Figure 6.

[0151] The system 100 estimates the stress level of each individual user and determines the user state 13 of each user 20 in the group of users 20 based on the estimated stress level.

[0152] External factor 14 is the splitting of the user group 20. Mobile robot 1 can detect the splitting of the group by monitoring the group with its camera sensor. System 100 may determine external factor 14 by evaluating the information provided by the mobile robot 1's camera sensor.

[0153] If system 100 determines that some users 20 in a group of users 20 have a high stress level and predicts that the users 20 in the group are unwilling to wait for each other, system 100 adapts the behavior plan of the improved behavior planner 19. The behavior plan adaptation means 8 enables a behavior that includes "waiting for all users 20 by stopping in a safe area," and the improved behavior planner 19 takes this additional behavior option into consideration when planning the behavior 16 of the mobile robot 1.

[0154] The behavior planning adaptation means 8 may enable behaviors that include "slow down until the distance between the group of users 20 decreases." The improved behavior planner 19 then takes this additional behavior option into consideration when planning the behavior 16 of the mobile robot 1.

[0155] Figure 7 shows walking time as an external factor 14 in one application example.

[0156] The user 20 accompanying the mobile robot 1 may be in a user state 13 due to fatigue from prolonged walking, and may be determined to have the intention to rest before resuming walking.

[0157] The system 100 determines the user state 13 "fatigued user state" by measuring external factors 14 as the duration of walking or the distance walked, based on the pulse rate sensed by the user 20.

[0158] The system 100 may identify additional behavioral options to adapt the behavioral planning process of the mobile robot 1 by identifying suitable places for the user 20 and the mobile robot 1 to rest in their vicinity. In the example shown in Figure 7, the system identifies a coffee shop that is a short walking distance from the current position and is on the path to the walking target.

[0159] System 100 monitors the user state 13 of user 20, for example, the pulse rate measured by a smartwatch worn by user 20. If system 100 determines that the pulse rate exceeds a predetermined pulse rate threshold and the determined movement intention 12 of user 20 does not indicate that user 20 wants to rest, system 100 adjusts the parameters of the behavior planning process in the improved behavior planner 19 and enables additional behavior for planning the movement path of mobile robot 1. Additional behavior for behavior planning includes mobile robot 1 moving from its current position to a specified rest location, for example, a coffee shop in the example of Figure 7, together with user 20. Additional behavior for behavior planning includes mobile robot 1 recommending to user 20 via the human-machine interface 5 that they go to the specified rest location.

[0160] Figure 8 shows the processing of the behavior planner 19 in the first example.

[0161] System 100 determines the user state 13 based on information about the user 20's physical condition, using sensor information including, for example, the perceived heart rate obtained by a smartwatch worn by the user 20. The external factor determination means 6 then accesses and searches the database 7 based on the determined user state 13 to determine at least one relevant external factor 14. The behavior planner fitting means 8 searches the database 10 using the determined user state 13, for example, the heart rate, and a list of external factors 14, for example, the user 20's speed, the user 20's past curves, shaded areas in the environment, and road surface characteristics, to determine the fitting of the total cost used in the behavior planner 9. Fitting the total cost of the behavior planner may include additional cost components in the total cost obtained from the database 10 based on the user state 13, the external factor 14, and the list of costs stored in the database 10 associated with each user state 13 and external factor 14.

[0162] The database 10 stores multiple cost components associated with the user state 13 and external factors 14.

[0163] The left portion of Figure 8 shows that at time t, the system 100 determines that the determined user state 13 exceeds the threshold level. Subsequently, the system 100, in particular the behavior planner adaptation means 8, HS>HS thr,1 , and EF > EF thr,1 (3) Evaluate this, and if equation (3) is true, then add the cost component C to the total cost of the behavior planner. In equation (3), HS represents user state 13, and HS thr,1 represents the threshold for user state 13. EF represents external factors 14, and EF thr,1 represents the threshold for external factor 14. Alternatively or additionally, if equation (3) is true, the behavior planner fitting means 8 corrects the current weight of cost component C.

[0164] The core behavior planner 9 of the improved behavior planner 19 then plans the behavior 16 of the mobile robot 1 using the fitted total cost, which includes at least one of the added cost component C and the corrected cost weights of the cost component C.

[0165] Figure 9 shows the processing in the behavior planner 19, and in particular the behavior planner adaptation means 8, in the second example.

[0166] The database 10 stores multiple behaviors of the mobile robot 1 associated with user status 13 and external factors 14.

[0167] The left portion of Figure 9 shows that at time t, the system 100 determines that the determined user state 13 exceeds the threshold level. The system 100, in particular the behavior planner adaptation means 8, then performs equation (4) HS>HS thr,1 , and EF > EF thr,1 (4) Evaluate this, and if equation (4) is true, then enable the individual behavior B of the behavior planner 19. In equation (4), HS represents the user state 13, and HS thr,1 represents the threshold for user state 13. EF represents external factors 14, and EF thr,1 represents the threshold for external factor 14. The behavior planner adaptation means 8 then provides the enabled individual behavior (fallback behavior B) to the behavior planner 19. The core behavior planner 9 of the improved behavior planner 19 plans the behavior 16 of the mobile robot 1, taking into account the enabled individual behavior B as well.

[0168] Figure 10 shows a high-level overview diagram of a hardware architecture suitable for implementing System 100.

[0169] The system 100 for controlling the mobile robot 1, which is configured to accompany a human user 20 as shown in Figure 10, is implemented using the mobile robot 1's hardware resources, at least one sensor 49 located separately from the mobile robot 1, and a remote server 50. A communication network 40 provides communication between independent components of the system 100, including the mobile robot 1, the at least one sensor 49, and the remote server 50.

[0170] At least one sensor 49 may include a wearable device, such as a smartwatch worn by the user 20.

[0171] Server 50 may provide data storage capabilities for storing data from the first and second databases 7 and 10. Additionally or alternatively, the memory 43 of the mobile robot 1 may store the databases 7 and 10 as described with reference to the functional block diagram in Figure 2.

[0172] The mobile robot 1 comprises at least one processor 42 and memory 43. The processor 42 provides hardware for executing software, in particular, a plurality of software modules that implement an improved behavior planner 19, which includes acquisition means for acquiring information about the user, prediction means 12, user information acquisition means for acquiring information about the physiological state of the user 20, user state determination means 13, external factor determination means 6, and a core behavior planner 9 and behavior planner adaptation means 8.

[0173] The mobile robot 1 is equipped with at least one sensor 45, for example, a camera sensor 45 that monitors the environment of the mobile robot 1. At least one sensor 45 may provide information acquired by acquisition means for acquiring information about the user 20 and user information acquisition means for acquiring information about the user's physiological state.

[0174] As an alternative and additional, an acquisition means for acquiring information about the user 20 and a user information acquisition means for acquiring information about the user's physiological state, both running on the processor 42, acquire their respective information from at least one external sensor 49 via a communication network 40 using the communication means 41 of the mobile robot 1.

[0175] The mobile robot 1 is further equipped with a human-machine interface 46, which corresponds to the human-machine interface 5 described with reference to Figure 2 and performs the corresponding functions.

[0176] The mobile robot 1 is further equipped with a controller 47, which corresponds to the controller 4 described with reference to Figure 2 and performs the corresponding functions, in particular controlling the movement of the mobile robot 1. In detail, the controller 47 controls the actuators 48 of the mobile robot 1.

[0177] The robot's actuator 48 may include wheels, tracks, or legs of a humanoid robot that are driven by electric motors.

[0178] The mobile robot 1 may also include a robotic arm having an end effector as an actuator 48 controlled by a controller 47. The controller 47 may control the robotic arm to give directions to the user 20, remove obstacles from the planned travel path of the mobile robot 1 and the user 20, and use pointing gestures to guide the user 20 accompanying the mobile robot 1.

[0179] The communication interface 41 provides the mobile robot 1 with the ability to communicate via the communication network 40. The communication network may preferably provide communication via wireless communication.

[0180] The mobile robot in Figure 10 is equipped with a communication bus 44 for connecting the individual components of the mobile robot 1 to each other.

[0181] All steps performed by the various entities described in this disclosure, as well as functions performed by the various entities, mean that each entity is adapted or configured to perform its respective step and function.

[0182] In the claims and detailed description, the phrase "equipped with" does not preclude the presence of other elements or steps. The indefinite articles "a" and "an" do not preclude the plural form.

[0183] A single element or other component may perform the functions of several entities or items described in the claims. The mere fact that specific means and functions of a control circuit are described in different dependent claims does not preclude the fact that those means and functions cannot be combined in a favorable implementation.

Claims

1. A computer implementation method for controlling a mobile robot (1) that accompanies at least one user (20), Obtaining information about at least one user (20) (S1), Based on the acquired information relating to at least one user (20), predict the intention of the user's (20) actions (S2), The system acquires information regarding the physiological state of at least one user (20) (S3), and determines the user state based on the acquired information regarding the physiological state (S4), Determining at least one external factor that affects the user state determined above (S6), The behavior of the mobile robot (1) is planned (S7) to maximize the user comfort index determined based on the determined user state and the determined at least one external factor, A computer implementation method comprising generating a control signal for controlling the mobile robot (1) to perform the planned behavior, and controlling the mobile robot (1) based on the generated control signal (S8).

2. The computer implementation method according to claim 1, wherein the information relating to the physiological state includes information relating to at least one of the pulse rate, heart rate, blood oxygen concentration, blood pressure, and body temperature of the at least one user (20).

3. The computer implementation method according to claim 1, wherein determining the user state is further based on image information of the face of at least one user (20) or image information of the body shape of at least one user (20).

4. Determining the user state based on the information relating to the physiological state of at least one user (20) is done from at least one sensor (45) of the mobile robot (1), The computer implementation method according to claim 1, further comprising sensor information acquired from a camera sensor that captures image information representing at least one user (20).

5. The aforementioned at least one external factor, The computer implementation method according to claim 1, comprising at least one of crowd density, walking time, walking distance, spatial density of user groups, ambient temperature, shaded areas in the environment, sunlit areas in the environment, speed, number and arrangement of curves along the travel path.

6. Determining the aforementioned external factors Detecting rainfall or snowfall using a membrane capacity rain sensor or camera sensor. The ambient temperature is detected by a temperature sensor. The camera sensor is used to determine the spatial distribution of sunlight intensity, particularly the sunlight intensity within the environment of the mobile robot (1). The light sensor detects bright light. The wind sensor detects at least one of the wind direction and wind strength. A camera sensor captures an image of the surface, and a processor performs image processing on the captured image to determine at least one of the following: a rocky surface, a rough surface, a wet surface, a dirty road surface, or a road gradient. The server (50) retrieves at least one of the following from the database (10) stored in the server (50): traffic congestion information, the number of visitors to a store or museum, and waiting times. The communication unit (41) of the mobile robot (1) acquires sensor information for determining the at least one external factor. A computer implementation method according to claim 1, comprising at least one of the following.

7. The computer implementation method according to claim 1, wherein predicting the intent of the movement of the at least one user (20) (S2) includes predicting the user's path using a constant speed model based on the acquired information relating to the at least one user (20).

8. Planning the behavior of the mobile robot (1) means that if the user comfort index of the planned path exceeds the user comfort index of the intended movement path (25) of the intended movement of the at least one user (20), then planning the path of the mobile robot (1) to guide the at least one user (20) on a different movement path (24) than the intended movement path (25) of the expected movement of the at least one user (20), or If the user comfort index of the planned path does not exceed the comfort index of the intended movement path of the user (20) whose predicted movement intention, then the path of the mobile robot (1) to guide the user (20) along the intended movement path of the at least one user whose predicted movement intention is to be planned. The computer implementation method according to claim 1, including the method described in claim 1.

9. Planning the behavior of the mobile robot (1) The discomfort index is determined based on the user state determined and the at least one external factor, and if the determined discomfort index exceeds a threshold, The parameters in the behavior planner (19) are adapted according to the determined user state, the user's (20) intended movement, and the at least one external factor. The computer implementation method according to claim 1, including the method described in claim 1.

10. To adapt the parameters of the behavior planner (19), To adapt the cost components of the total cost term used in the aforementioned behavior planner (19), To add behavioral options for the aforementioned mobile robot (1), To adapt the weights of the cost components used in the behavior planner (19), The computer implementation method according to claim 9, comprising at least one of the following.

11. Planning the behavior of the mobile robot (1) Cost-based planners, especially collaborative risk mapping planners, or A search-based planner, for example A ＊ Alternatively, a planner based on Monte Carlo tree search, Sampling-based planners, such as RRT-based or probabilistic roadmap-based planners, Reinforcement learning, for example, a planner based on Q-learning or PPO The computer implementation method according to claim 1, comprising planning the aforementioned behavior accordingly.

12. Controlling the mobile robot (1) based on the generated control signal is The controllers (4, 47) control the movement of the mobile robot (1), and Based on the planned behavior, output information is generated, and this output information is output by the human-machine interface (5, 46) of the mobile robot (1). The computer implementation method according to claim 1, comprising at least one of the following.

13. A system for controlling a mobile robot (1) configured to accompany at least one user (20), An acquisition means for obtaining information about at least one user (20), Prediction means (2) for predicting the intent of the movements of the at least one user (20) based on the acquired information relating to the at least one user (20), User information acquisition means for obtaining information regarding the physiological state of at least one user (20), A user state determination means (3) that determines the user state based on the acquired information regarding the physiological state, Factor determination means (6) for determining at least one external factor that affects the determined user state, A behavior planner (19) plans the behavior of the mobile robot (1) in order to maximize the user comfort index determined based on the determined user state and at least one determined external factor, A controller (4) for generating a control signal to perform the planned behavior and controlling the actuator (48) of the mobile robot (1) based on the generated control signal, and A human-machine interface (5) for generating output information based on the planned behavior of the mobile robot (1) and outputting the generated output information to at least one user (20). A system comprising at least one of the following.

Images