Guide robot, control server, and method for controlling guide robot

Abstract

To selectively provide contents that suit expected actions based on positions of a guide robot and a user.SOLUTION: A guide robot according to the present invention comprises: a position recognition unit configured to recognize a position of a guide robot; a video recognition unit configured to recognize a user from a target image acquired from an image sensor; an action control unit configured to, in response to a call from the user, transmit, to a content recommendation unit, a request for generating a target content list in which the position of the guide robot and the characteristics of the user are reflected, and, upon receiving a target content list, output the target content list via an output unit of the guide robot; and a content recommendation unit configured to combine the position of the guide robot and map information to generate first feature data, generate second feature data of the user from the target image, and transmit, to the action control unit, the target content list acquired by applying each of the first feature data and the second feature data to a content recommendation model.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a guide robot, a control server, and a method for controlling a guide robot, and more particularly, to a technology for providing content to a user via a guide robot. [Background technology]

[0002] The guide robot can provide the user with content that can be used for services within the area in which it can move. The guide robot can interact with the user within the area in which it can move and provide the user with desired functions. As the area in which the guide robot can move is expanded, the functions required of the guide robot also increase. In this case, the manager of the guide robot adds new functions to the guide robot in order to increase the user's utilization of the robot. However, since it is not known what kind of user will use the guide robot, adding many functions and encouraging the user to select and use them may cause a problem in that unnecessary functions depending on the user are exposed, reducing the usability of the robot.

[0003] To solve these problems, it is necessary to develop technology that can map the area a guide robot can move around in, collect data that can determine the characteristics of the user, and combine this with the guide robot's current location and direction information to prioritize the content that the user expects from the robot. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-276080 A Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention aims to provide a guide robot, a control server, and a guide robot control method that can provide a user with a target content list reflecting data acquired from an image recognition unit and a position recognition unit, thereby selectively providing content that matches the position of the guide robot within the service area of ​​the guide robot and the expected behavior of the user based on the user's position, thereby effectively providing content to the user.

[0006] Another object of the present invention is to provide a guide robot, a control server, and a guide robot control method that update content included in map information by comparing content stored in a control server with content included in map information, thereby eliminating the need to provide unnecessary content to a user and increasing usability of the guide robot for a user.

[0007] In addition, the present invention aims to provide a guide robot, a control server, and a guide robot control method that efficiently distribute the amount of data calculation and provide customized content for a user by separating a content recommendation unit that acquires a target content list reflecting the location of the guide robot and the characteristics of a user from a data management unit that stores and manages the content, and that provides various contents according to the location of the same guide robot, thereby increasing the utilization rate of the robot. The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0008] According to an embodiment of the present invention, the guide robot includes a location recognition unit that recognizes a location of the guide robot; an image recognition unit that recognizes a user from a target image acquired from an image sensor; a behavior control unit that transmits a request for generating a target content list reflecting the location of the guide robot and characteristics of the user to a content recommendation unit in response to a call from the user, and outputs the target content list through an output unit of the guide robot based on the received target content list from the content recommendation unit; and the content recommendation unit that generates first feature data by combining the location of the guide robot with map information, generates second feature data related to characteristics of the user from the target image, and transmits the target content list obtained by applying each of the first feature data and the second feature data to a content recommendation model to the behavior control unit.

[0009] In one embodiment, the location recognition unit may acquire a previous time point, which is a time point close to the target time point and at which the guide robot has moved, from among previous time points based on the target time point at which the location of the guide robot is recognized, and may acquire a direction of the guide robot at the target time by combining the location of the guide robot at the previous time point and the location of the guide robot at the target time point with the map information, and transmit at least one of the location of the guide robot at the target time point, the direction of the guide robot at the target time point, or any combination thereof, to the content recommendation unit.

[0010] In an embodiment, the content recommendation unit generates the first feature data including at least one of a position of the guide robot at the target time point, a direction of the guide robot at the target time point, or any combination thereof, sorts at least one content included in the map information in order of proximity to the position of the guide robot included in the first feature data, and applies the sorted content and the first feature data to the content recommendation model to obtain a first sub-content list including content related to a direction opposite to the direction of the guide robot, and the first sub-content list may be included in the target content list.

[0011] In one embodiment, the image recognition unit recognizes at least one of the user's gender information, age information, face information, or any combination thereof from the target image, generates information regarding tracking of the face information based on the recognition of the face information from the target image, and transmits at least one of the recognized information, the tracking information, or any combination thereof to the content recommendation unit, and the content recommendation unit generates the second feature data including at least one of the recognized information, the tracking information, or any combination thereof, applies the second feature data to the content recommendation model, and obtains a second sub-content list including content that reflects the user's characteristics from at least one content included in the map information, and the second sub-content list may be included in the target content list.

[0012] In one embodiment, the guide robot further includes a data management unit that updates at least one content included in the map information, and the data management unit transmits a version of each of the at least one content included in the map information to a control server, updates a content among the versions of the at least one content included in the map information that is different from a version of the content stored in the control server, and transmits an update failure notification to the behavior control unit when the update of the content included in the map information that is different from the version of the content stored in the control server fails more than a predetermined number of times.

[0013] In an embodiment, the data management unit may obtain an initial position of the guide robot based on a time when power was applied to the guide robot based on the time when power was applied to the guide robot being less than a predetermined first threshold, update content that is included in the map information and different from a content version stored in the control server based on the guide robot being stopped at the initial position, and transmit a request to the behavior control unit to move the guide robot to a predetermined standby position based on the update failure notification transmitted to the behavior control unit.

[0014] In one embodiment, the data management unit may acquire a duration of the idle state of the guide robot based on the state of the guide robot being idle, update content that is included in the map information and different from the version of the content stored in the control server based on the duration exceeding a predetermined second threshold, and transmit a request to the behavior control unit to move the guide robot to a predetermined standby position based on the update failure notification transmitted to the behavior control unit.

[0015] In one embodiment, the data management unit may acquire a charge amount of the guide robot based on the state of the guide robot being in a charging state, update content that is included in the map information and different from the content version stored in the control server based on the charge amount being equal to or greater than a third predetermined threshold, and transmit a request to the behavior control unit to maintain charging of the guide robot based on the update failure notification transmitted to the behavior control unit.

[0016] In one embodiment, the content recommendation unit transmits data regarding a graphic object in which each of the contents included in the target content list is displayed as a predetermined area to the behavior control unit based on receiving resources related to the target content list from a control server.

[0017] A control server according to an embodiment of the present invention includes a memory storing computer-executable instructions, at least one processor accessing the memory and executing the instructions, and a communication unit communicating with a guide robot, the at least one processor generating map information including a location where the guide robot is located and including content that the guide robot can guide a user to, extracting common content that can be provided to the user in a first area based on the location where the guide robot is located from the content stored in the control server, and transmitting the map information, the extracted common content, and the extracted specialized content to the guide robot in response to a request from the guide robot.

[0018] In one embodiment, the at least one processor generates a guide map in which the extracted content is displayed as a predetermined area based on the map information, and transmits the guide map to the guide robot in response to a resource request from a content recommendation unit of the guide robot.

[0019] A method for controlling a guide robot according to an embodiment of the present invention includes an operation of recognizing a position of a guide robot, an operation of recognizing a user from a target image acquired from an image sensor, an operation of transmitting a request for generating a target content list reflecting the position of the guide robot and characteristics of the user to a content recommendation unit in response to a call from the user, and an operation of outputting the target content list through an output unit of the guide robot based on receiving the target content list from the content recommendation unit, an operation of generating first feature data by combining the position of the guide robot with map information, and generating second feature data related to characteristics of the user from the target image, and an operation of applying each of the first feature data and the second feature data to a content recommendation model to transmit the target content list obtained to a behavior control unit.

[0020] In an embodiment, the operation of recognizing the position of the guide robot may include an operation of acquiring a previous time point, which is a time point close to the target time point and at which the guide robot has moved, among previous time points based on the target time point at which the position of the guide robot is to be recognized, an operation of acquiring a direction of the guide robot at the target time point by combining the position of the guide robot at the previous time point and the position of the guide robot at the target time point with the map information, and an operation of transmitting at least one of the position of the guide robot at the target time point, the direction of the guide robot at the target time point, or any combination thereof to the content recommendation unit.

[0021] In an embodiment, the guide robot control method may include an operation of generating the first feature data including at least one of a position of the guide robot at the target time point, a direction of the guide robot at the target time point, or any combination thereof; an operation of sorting at least one content included in the map information in order of proximity to a straight-line distance to the position of the guide robot included in the first feature data; and an operation of applying the sorted content and the first feature data to the content recommendation model to obtain a first sub-content list to be included in the target content list, the first sub-content list including content in an opposite direction to a direction of the guide robot.

[0022] In one embodiment, the operation of recognizing the user may include an operation of recognizing at least one of the user's gender information, age information, face information, or any combination thereof from the target image, an operation of generating information regarding tracking of the face information based on the recognition of the face information from the target image, an operation of transmitting at least one of the recognized information, the tracking information, or any combination thereof to the content recommendation unit, an operation of generating the second feature data including at least one of the recognized information, the tracking information, or any combination thereof, and an operation of applying the second feature data to the content recommendation model to obtain a second sub-content list to be included in the target content list, the second sub-content list including content reflecting characteristics of the user from at least one content included in the map information.

[0023] In one embodiment, the guide robot control method may include an operation of transmitting each version of at least one content included in the map information to a control server, an operation of updating a content, which is different from a version of the content included in the map information and stored in the control server, among each version of the at least one content included in the map information, and an operation of transmitting an update failure notification to the behavior control unit based on the fact that an update of the content, which is included in the map information and is different from the version of the content stored in the control server, has failed a predetermined number of times.

[0024] In one embodiment, the operation of performing the update may include an operation of acquiring an initial position of the guide robot based on a time when power was applied to the guide robot based on the time when power was applied to the guide robot being less than a predetermined first threshold, an operation of updating content that is included in the map information and differs from a version of the content stored in the control server based on the guide robot stopping at the initial position, and an operation of transmitting a request to the behavior control unit to move the guide robot to a predetermined standby position based on the transmission of the update failure notification to the behavior control unit.

[0025] In one embodiment, the operation of performing the update may include an operation of acquiring a duration of an idle state of the guide robot based on the state of the guide robot being an idle state, an operation of updating content that is included in the map information and that differs from a version of the content stored in the control server based on the duration exceeding a predetermined second threshold, and an operation of transmitting a request to the behavior control unit to move the guide robot to a predetermined standby position based on the transmission of the update failure notification to the behavior control unit.

[0026] In one embodiment, the operation of performing the update may include an operation of acquiring a charge amount of the guide robot based on the state of the guide robot being in a charging state, an operation of updating content that is included in the map information and that differs from a content version stored in the control server based on the charge amount being equal to or greater than a third predetermined threshold, and an operation of transmitting a request to the behavior control unit to maintain charging of the guide robot based on the transmission of the update failure notification to the behavior control unit.

[0027] In one embodiment, the guide robot control method may further include an operation of generating map information including a location where the guide robot is located and including content that the guide robot can guide a user to; an operation of extracting common content that can be provided to the user in a first area based on the location where the guide robot is located, and specialized content that can be provided to the user in a second area based on the location where the guide robot is located, from the content stored in a control server; and an operation of transmitting the map information, the extracted common content, and the extracted specialized content to the guide robot in response to a request from the guide robot. Effect of the Invention

[0028] The advantages of the guide robot, control server, and guide robot control method according to the present invention are as follows: According to at least one of the embodiments of the present invention, a target content list reflecting data acquired from the image recognition unit and the position recognition unit is provided to a user, thereby selectively providing contents that match the position of the guide robot within the service area of ​​the guide robot and the expected behavior of the user based on the position of the user, thereby providing contents to the user effectively.

[0029] In addition, according to at least one embodiment of the present invention, by comparing the content stored in the control server with the content included in the map information and updating the content included in the map information, it is possible to avoid providing unnecessary content to the user, thereby increasing the usability of the guide robot for the user.

[0030] In addition, according to at least one embodiment of the present invention, a content recommendation unit that acquires a target content list reflecting the location of the guide robot and the characteristics of a user and a data management unit that stores and manages the content are separated, so that the amount of data calculation can be efficiently distributed, and content specialized for a user can be provided, and various contents can be provided according to the location of the same guide robot, thereby increasing the utilization rate of the robot. In addition, various effects that can be directly or indirectly understood through this document can be provided. [Brief description of the drawings]

[0031] [Figure 1] FIG. 2 illustrates a guide robot according to an embodiment of the present invention. [Diagram 2] 4 is a flowchart illustrating a method for outputting a list of target contents in a guide robot according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a detailed view of a guide robot according to an embodiment of the present invention. [Figure 4] 5 is a diagram illustrating a process of outputting a list of target contents in response to a user call in the guide robot according to an embodiment of the present invention. [Diagram 5] 11 is a diagram for explaining an operation of a data management unit in the guide robot according to the embodiment of the present invention. FIG. [Figure 6] 5A to 5C are diagrams for explaining the operation of a behavior control unit in the guide robot according to the embodiment of the present invention. [Figure 7] 5 is a diagram illustrating the operation of a video recognition unit and a position recognition unit in the guide robot according to the embodiment of the present invention. FIG. [Figure 8a] 5 is a flowchart illustrating an operation of a data management unit updating contents in the guide robot according to an embodiment of the present invention. [Figure 8b] 5 is a flowchart illustrating an operation of a data management unit updating contents in the guide robot according to an embodiment of the present invention. [Figure 8c] 5 is a flowchart illustrating an operation of a data management unit updating contents in the guide robot according to an embodiment of the present invention. [Figure 9] 4 is a flowchart illustrating a method for outputting a target content list in a guide robot according to an embodiment of the present invention. [Figure 10] FIG. 2 illustrates a control server according to an embodiment of the present invention. [Figure 11] 11 is a diagram illustrating a method of transmitting map information and a content list to a guide robot in a control server according to an embodiment of the present invention. [Figure 12] 11 is a diagram showing a guide map generated based on map information in a control server according to one embodiment of the present invention. FIG. [Figure 13] FIG. 4 is a diagram for explaining a source code of a guide map according to an embodiment of the present invention. [Figure 14] In the drawings, the same or similar reference numerals are used for the same or similar components, and the same or similar components are used for the same or similar components, and the same or similar components are used for the same or similar components, in accordance with an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] Hereinafter, some embodiments of the present invention will be described in detail with reference to the exemplary drawings.

[0033] When adding reference symbols to components in each drawing, the same symbols are used for the same components as much as possible. In addition, when describing the embodiments of the present invention, detailed descriptions of related known structures or functions will be omitted if they hinder understanding of the embodiments of the present invention. In particular, various embodiments will be described with reference to the accompanying drawings. However, this is not intended to limit the technology described in this document to a specific embodiment, and it should be understood that the embodiments of the present document include various modifications, equivalents, and / or alternatives. In connection with the description of the drawings, similar reference symbols may be used for similar components.

[0034] In describing the components of the embodiments of the present invention, terms such as 1, 2, A, B, (a), (b) and the like may be used. Such terms are used to distinguish the components from other components, and do not limit the essence, order, or sequence of the components. In addition, unless otherwise defined, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by those having ordinary skill in the art to which the present invention belongs. Terms similar to those defined in commonly used dictionaries must be interpreted as meanings consistent with the meanings they have in the context of the relevant art, and unless otherwise clearly defined in this application, they should not be interpreted in an idealized and overly formal sense.

[0035] For example, terms such as "first," "second," "primary," or "secondary" used herein may modify various components regardless of order and / or importance and are used only to distinguish one component from another component and do not limit the components. For example, a first user device and a second user device refer to different user devices regardless of order or importance. For example, a first component may be named a second component, and similarly, a second component may be named instead of a first component, without departing from the scope of the rights described herein.

[0036] In this document, expressions such as "having," "may have," "including," or "may include" indicate the presence of a feature (e.g., a value, function, operation, or component) and do not exclude the presence of additional features. When a component (e.g., a first component) is referred to as being "operatively or communicatively coupled with / to" or "connected to" another component (e.g., a second component), it should be understood that the component is directly coupled to the other component or is coupled through another component (e.g., a third component). On the other hand, when a component (e.g., a first component) is referred to as being "directly coupled" or "directly connected" to another component (e.g., a second component), it should be understood that there is no other component (e.g., a third component) between the component and the other component.

[0037] As used in this document, the expression "configured to" may be used alternatively with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of," depending on the context.

[0038] The term "configured to" does not necessarily mean "specifically designed to" hardware. Instead, depending on the context, an "apparatus configured to" may mean that the apparatus is "capable of" performing, in conjunction with other devices or components. For example, the phrase "a processor configured to perform A, B, and C" may refer to a dedicated processor for performing those operations (e.g., an embedded processor) or a general-purpose processor (e.g., a CPU or application processor) that performs those operations by executing one or more software programs stored in a memory device.

[0039] The terms used in this document are merely used to describe a particular embodiment and may not be intended to limit the scope of other embodiments. A singular expression may include a plural expression unless the context clearly indicates otherwise. The terms used herein, including technical and scientific terms, may have the same meaning as commonly understood by a person having ordinary skill in the technical field described in this document. Among the terms used in this document, terms defined in a general dictionary may be interpreted as having the same or similar meaning as the meaning they have in the context of the related art, and unless clearly defined in this document, they are not interpreted as being ideal and overly formal. In some cases, even terms defined in this document cannot be interpreted as excluding the embodiments of this document.

[0040] In this document, phrases such as "A or B," "at least one of A and / or B," or "one or more of A and / or B" may include all possible combinations of the items listed together. For example, "A or B," "at least one of A and B," or "at least one of A or B" may refer to all cases in which (1) at least one A is included, (2) at least one B is included, or (3) both at least one A and at least one B are included.

[0041] In addition, in describing components of the embodiments of the present invention, each of the terms "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B, or C," and "at least one of A, B, C, or some combination thereof" may include any one of the items listed with the term, or all possible combinations thereof. In particular, a term such as "at least one of A, B, C, or some combination thereof" may include A, B, or C, or combinations thereof such as AB or ABC.

[0042] Hereinafter, an embodiment of the present invention will be described in detail with reference to FIGS.

[0043] 1 is a diagram illustrating a guide robot according to an embodiment of the present invention. The guide robot 100 according to an embodiment may include an image recognition unit 110, a location recognition unit 120, a behavior control unit 130, a content recommendation unit 140, and a communication unit 150 that supports communication between the guide robot 100 and a control server 160. The guide robot 100 according to an embodiment may omit at least one of the components or may additionally include other components.

[0044] The guide robot 100 is a robot that responds to a call from a user and outputs a list of target contents reflecting the location of the guide robot 100 and the characteristics of the user. For example, the guide robot 100 recognizes the user from a target image acquired from an image sensor by the image recognition unit 110. The guide robot 100 recognizes the location of the guide robot 100 by the location recognition unit 120. The guide robot transmits data recognized by the image recognition unit 110 and the location recognition unit 120 (e.g., the location of the user or the guide robot 100) to the behavior control unit 130 and the content recommendation unit 140.

[0045] The guide robot 100 performs the following operations to output a list of target contents in response to a user's call. For example, the guide robot 100 transmits a request for generating a list of target contents reflecting the location of the guide robot 100 and the characteristics of the user to the content recommendation unit 140 through the behavior control unit 130.

[0046] The guide robot 100 generates first feature data by combining the location of the guide robot 100 with map information through the content recommendation unit 140, and generates second feature data related to the user's characteristics from the target image. The guide robot 100 applies each of the first feature data and the second feature data to a content recommendation model (e.g., not shown) to obtain a target content list, and transmits the obtained target content list to the behavior control unit 130. The guide robot 100 outputs the target content list through the output unit through the behavior control unit 130. A detailed description of the content recommendation model that receives the first feature data and the second feature data and generates a target content list will be described later with reference to FIG. 2.

[0047] For reference, the guide robot 100 performs all the operations performed by the image recognition unit 110, the position recognition unit 120, the behavior control unit 130, and the content recommendation unit 140. Therefore, for convenience of explanation, in this specification, the operations performed by the image recognition unit 110, the position recognition unit 120, the behavior control unit 130, and the content recommendation unit 140 are mainly explained as operations performed by the guide robot 100. Also, detailed explanations of the image recognition unit 110, the position recognition unit 120, the behavior control unit 130, and the content recommendation unit 140 will be given later with reference to FIG. 3 below.

[0048] FIG. 2 is a flow chart illustrating a method for outputting a target content list in a guide robot according to an embodiment of the present invention. In operation 210, a guide robot (e.g., the guide robot 100 of FIG. 1) according to an embodiment generates first feature data by combining the location of the guide robot with map information, and generates second feature data related to the user's characteristics from the target image. For example, the first feature data may include information on the location of the guide robot and the direction in which the guide robot is heading as feature data related to the guide robot. In particular, the direction in which the guide robot is heading is obtained based on the location of the guide robot and map information. A detailed description of the direction in which the guide robot is heading will be described later with reference to FIG. 7. In contrast, the second feature data may include information recognized in an image obtained by an image recognition unit of the guide robot and tracking data related to the user's face as feature data related to the user. A detailed description of this will be described later with reference to FIG. 7.

[0049] In operation 220, the guide robot applies each of the first and second feature data to the content recommendation model to obtain a target content list, and transmits the list to the behavior control unit.

[0050] For example, the content recommendation model may include a neural network model. Specifically, at least one of the guide robot, the control server (e.g., the control server 160 in FIG. 1), or other learning devices may train the content recommendation model. The neural network may include multiple layers, and each layer may include multiple nodes. The nodes may have node values ​​determined based on an activation function. Nodes in any layer are connected to nodes (e.g., other nodes) in other layers via links (e.g., connection edges) having connection weights. The node values ​​of the nodes are propagated to other nodes via the links. In the inference operation of the neural network, the node values ​​are forward propagated from the previous layer to the next layer.

[0051] Exemplarily, in the content recommendation model, a forward propagation operation refers to an operation of propagating a node value based on input data from an input layer to an output layer of the content recommendation model. That is, the node value of a node is propagated (e.g., forward propagation) to a node (e.g., a next node) in a subsequent layer connected to the node via a connection line. For example, a node receives a value weighted by a connection weight value from a previous node (e.g., a plurality of nodes) connected to the node via a connection line.

[0052] The node value of a node is determined based on applying an activation function to a sum (e.g., weighted sum) of weighted values ​​received from previous nodes. The parameters of the neural network may illustratively include the connection weights described above. The parameters of the neural network are updated such that the objective function value described below is changed in a targeted direction (e.g., a direction in which the loss is minimized).

[0053] The trained content recommendation model may indicate a model trained by machine learning, and may be a trained machine learning model that outputs a training output (e.g., a target content list) from a training input (e.g., first feature data and second feature data). However, the training input and the training output are not limited thereto. For example, the trained content recommendation model obtains a first sub-content list including contents related to the opposite direction of the guide robot from the aligned content and the first feature data. For reference, a detailed description of the aligned content will be provided below with reference to FIG. 4.

[0054] Similarly, the trained content recommendation model obtains a second sub-content list including contents reflecting the user's characteristics from the second feature data.

[0055] The guide robot acquires a target content list by combining the first subcontent list and the second subcontent list acquired by the content recommendation model based on a predetermined priority. For example, if the predetermined priority is related to the location of the guide robot, the guide robot acquires a target content list in which the first subcontent list, which has priority over the second subcontent list, is combined. Also, if the predetermined priority is related to the characteristics of the guide user, the guide robot acquires a target content list in which the second subcontent list, which has priority over the first subcontent list, is combined.

[0056] The machine learning model (e.g., the trained content recommendation model) is generated by machine learning. The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the aforementioned examples.

[0057] The trained machine learning model may include multiple artificial neural network layers. Specifically, the artificial neural network may be at least one combination of a deep neural network (DNN), a convolutional neural network (CNN), a U-Net for Image Segmentation (U-net), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRRDNN), a deep Q-network, or a combination thereof, but is not limited to the above examples.

[0058] In the case of supervised learning, the machine learning model is trained based on training data including a pair of a training input and a training output mapped to the training input. For example, the machine learning model is trained to output a training output from a training input. The machine learning model during training may generate a temporary output in response to the training input, and is trained to minimize a loss between the temporary output and a training output (e.g., a training target and correct answer data). During the training process, a parameter of the machine learning model (e.g., a connection weight value between nodes / layers in a neural network) is updated according to the loss. For example, such training may be performed by the guide robot itself where the machine learning model is performed, or may be performed via a separate server. The machine learning model that has completed the training (e.g., a trained content recommendation model) is stored in a memory of the guide robot. In operation 230, the guide robot outputs the target content list through an output unit of the guide robot based on receiving the target content list from the content recommendation unit. For example, the guide robot transmits a request for generating a target content list reflecting the location of the guide robot and the characteristics of the user to the content recommendation unit in response to a call from the user through the behavior control unit. The guide robot outputs the target content list via the output unit based on the received target content list from the content recommendation unit.

[0059] FIG. 3 is a detailed diagram of a guide robot according to an embodiment of the present invention. According to an embodiment, the guide robot 301 may include an input unit 303, a sensor 305, a control unit 307, and an output unit 321. According to an embodiment, the control server 323 may include a processor and a memory. For reference, the control server 323 will be described in detail below with reference to FIG. 10.

[0060] The input unit 303 transmits, for example, a command or data input from a user or another external device to the control unit 307. For example, the input unit 303 may include a touch sensor and a microphone. The input unit 303 receives a content extraction answer from a user or an external device through the touch sensor. Also, the input unit 303 receives a content extraction answer from a sound generated by the user or a sound generated from an external device through the microphone. The sensor 305, for example, recognizes information regarding the position of the guide robot 301, or recognizes the surrounding environment or the user of the guide robot 301. For example, the sensor 305 may include an image sensor and a position sensor.

[0061] The control unit 307, for example, acquires a target content list based on commands or data received by the input unit 303 and the sensor 305, and transmits the acquired target content list to the output unit 321. For example, the control unit 307 may include a UI input unit 309, a location recognition unit 311, an image recognition unit 313, a behavior control unit 315, a content recommendation unit 317, and a data management unit 319.

[0062] The UI input unit 309 receives commands or data input via the input unit 303. The UI input unit 309 transmits the received commands or data to the behavior control unit 315.

[0063] The position recognition unit 311 receives information about the position of the guide robot 301 recognized by a position sensor included in the sensor 305. The position recognition unit 311 obtains the direction of the guide robot 301 based on the target time point and the previous time point at which the position of the guide robot 301 was recognized. The position recognition unit 311 transmits at least one of the position of the guide robot 301 at the target time point, the direction of the guide robot 301 at the target time point, or any combination thereof, to the behavior control unit 315 and the content recommendation unit 317, respectively. This will be described in detail later with reference to FIG. 7.

[0064] The image recognition unit 313 receives information about the surrounding environment or the user recognized by an image sensor included in the sensor 305. The image recognition unit 313 recognizes and / or generates information about the user's gender information, age information, face information, and face information tracking information from a target image acquired from the image sensor. The image recognition unit 313 transmits at least one of the recognized information, tracking information, or any combination thereof to the behavior control unit 315 and the content recommendation unit 317, respectively. This will be described in detail later with reference to FIG. 7.

[0065] The behavior control unit 315 requests the content recommendation unit 317 to generate a target content list based on information received from the UI input unit 309, the location recognition unit 311, and the image recognition unit 313, or transmits the received information to the output unit 321. A detailed description of the behavior control unit 315 will be given later with reference to FIG. 6.

[0066] The content recommendation unit 317 acquires a target content list based on, for example, first feature data generated by combining the location of the guide robot 301 with map information and second feature data related to the user's characteristics generated from the target image. That is, the content recommendation unit 317 may be configured to provide and / or recommend content specialized for the user by acquiring the target content list.

[0067] The data management unit 319 may be configured to manage data required for the content recommendation unit 317 to generate a target content list or for the behavior control unit 315 to transmit a result (e.g., a target content list or a map on which the target content list is displayed) to the output unit 321. In particular, the data management unit 319 updates data stored in the guide robot 301 through communication with the control server 323. A method for updating data by the data management unit 319 will be described in detail later with reference to Figs. 8a to 8c below.

[0068] The output unit 321 may include a screen display unit, a motor drive unit, a wheel drive unit, and an audio output unit. The output unit 321 outputs data or information received from other components of the guide robot 301 to a user or other external devices. In particular, the screen display unit may include, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, a microelectromechanical systems (MEMS) display, or an electronic paper display. The screen display unit displays various contents (e.g., contents included in a target content list) to a user.

[0069] 4 is a diagram illustrating a process of outputting a list of target contents in response to a user call in the guide robot according to an embodiment of the present invention. The guide robot 400 according to an embodiment of the present invention may include a UI input unit 410, an image recognition unit 420, a location recognition unit 430, a behavior control unit 440, a content recommendation unit 450, a data management unit 460, and an output unit 470.

[0070] The behavior control unit 440 transmits a request for generating a content list to the content recommendation unit 450 based on receiving a call from the user from the UI input unit 410. The content recommendation unit 450 generates first feature data including at least one of a position of the guide robot 400 at a target time, a direction of the guide robot 400 at a target time, or any combination thereof. Here, the position of the guide robot 400 may be the position of the guide robot 400 recognized by the position recognition unit 430.

[0071] The content recommendation unit 450 sorts at least one content included in the map information in order of proximity to the position of the guide robot included in the first feature data. For example, the content recommendation unit 450 obtains at least one content included in the map information sorted in order of proximity to the position of the guide robot through the above-mentioned operation.

[0072] The content recommendation unit 450 applies the aligned contents and the first feature data to a content recommendation model to obtain a first subcontent list including contents related to the direction opposite to the direction of the guide robot 400. For example, the first subcontent list may be included in a target content list. Specifically, the first subcontent list may include contents that are close to the position of the guide robot 400 and can be provided by the guide robot 400 in the direction opposite to the direction of the guide robot 400 among the aligned contents.

[0073] The content recommendation unit 450 generates second feature data including at least one of the recognized information, the tracking-related data, or any combination thereof. Here, the recognized information and the tracking-related data may be data generated by the video recognition unit 420.

[0074] The content recommendation unit 450 applies the second feature data to the content recommendation model to obtain a second subcontent list including at least one content included in the map information that reflects the user's characteristics. For example, the second subcontent list may be included in the target content list, different from the first subcontent list. Specifically, the second subcontent list may include at least one content included in the map information that can be provided to the recognized user and / or recommended to the recognized user.

[0075] The content recommendation unit 450 generates a target content list based on the first subcontent list and the second subcontent list. For example, the content recommendation unit 450 requests resources from the control server 480 based on the generation of the target content list. Here, the resources indicate data that can output the contents included in the target content list as graphic objects through the output unit 470. Thereafter, the content recommendation unit 450 responds to a content list generation request from the action control unit 440 and transmits the target content list and related resources.

[0076] The behavior control unit 440 outputs the target content list via the output unit 470 based on receiving the target content list from the content recommendation unit 450. In this way, the behavior control unit 440 provides the user with specialized content (e.g., content included in the target content list).

[0077] 5 is a diagram for explaining the operation of a data management unit in a guide robot according to an embodiment of the present invention. A guide robot (for example, the guide robot 100 of FIG. 1) according to an embodiment may include a data management unit 500. For example, the data management unit 500 updates at least one content included in map information. Here, the map information may be data constituting a map that is provided to a user together with the content as map information stored in the guide robot. This will be described in detail later with reference to FIG. 12.

[0078] The data management unit 500 transmits each version of at least one content included in the map information to the second control server 520 of the control servers. The data management unit 500 updates the content, which is different from the version of the content stored in the second control server 520, among the versions of at least one content included in the map information. The data management unit 500 transmits an update failure notification to the behavior control unit when the update of the content, which is different from the version of the content included in the map information and stored in the second control server 520, has failed a predetermined number of times. A detailed description of the update of at least one content included in the map information will be described later with reference to FIGS. 8a to 8c.

[0079] According to an embodiment, the control server (for example, the control server 160 in FIG. 1) may include a first control server 510 and a second control server 520. For example, the first control server 510 is a server that manages a guide robot and manages map information stored in the guide robot or at least one piece of content included in the map information. In contrast, the second control server 520 is a server that provides content and provides content corresponding to attributes of the guide robot (for example, version, hardware specs, etc.) in response to a request from the data management unit 500.

[0080] The first control server 510 may include a map management module, a robot setting module, and a service area module. For example, the map management module refers to a module that manages map information stored in the guide robot. The robot setting module refers to a module that manages settings of the guide robot. The service area module refers to a module that manages content provided to a user along with map information. That is, the content is provided to a user in an area in which the guide robot can move. The operation of providing the content to a user is understood as the guide robot providing a service to a user.

[0081] The second control server 520 may include a contents discovery module and a contents module. For example, the contents discovery module refers to a module that searches for contents stored in the second control server 520 based on receiving a content update request from the data management unit 500. The contents module may include at least one content. Specifically, the contents module may include content related to vehicle explanations, content related to facility guides, and content related to current maintenance status. The content included in the contents module is transmitted to the guide robot by the data management unit 500.

[0082] FIG. 6 is a diagram for explaining the operation of the behavior control unit in the guide robot according to the embodiment of the present invention.

[0083] A guide robot (e.g., the guide robot 100 of FIG. 1) according to an embodiment may include a UI input unit 610, an image recognition unit 620, a location recognition unit 630, a behavior control unit 640, a content recommendation unit 650, and an output unit 660. The description of the UI input unit 610, the image recognition unit 620, the location recognition unit 630, and the output unit 660 may be the same as that described above, and therefore may be omitted in the description of FIG. 6. Hereinafter, the description will be mainly focused on the operation of the behavior control unit 640.

[0084] The behavior control unit 640 may include a UI input module, a behavior processing module, a UI output module, and a motion control module. For example, the UI input module receives data and / or information transmitted from the UI input unit 610, the image recognition unit 620, and the position recognition unit 630.

[0085] The motion processing module transmits a request for generating a content list to the content recommendation unit 650. Also, the motion processing module generates a motion of the guide robot for the content included in the target content list received from the content recommendation unit 650. For example, in the case of content related to vehicle guidance, the motion processing module generates a motion of the guide robot guiding the vehicle to the user and a motion of providing the content related to vehicle guidance. As a result, the motion processing module transmits the generated motion to each of the UI output module and the motion control module.

[0086] The motion control module controls at least one of the neck, arms, wheel drive, or any combination of these of the guide robot. The UI output module outputs the contents included in the target content list to the user through a screen display unit and a voice output unit attached to the guide robot.

[0087] In relation to the UI output module outputting the contents included in the target content list through the screen display unit, the UI output module transmits resources related to the target content list to the screen display unit. For example, based on receiving the resources related to the target content list from the control server, the content recommendation unit 650 transmits data related to graphic objects in which each of the contents included in the target content list is displayed as a predetermined area to the behavior control unit 640. In particular, the content recommendation unit 650 transmits the data related to the graphic objects to the action processing module of the action control unit 640, and the action processing module transmits the transmitted data related to the graphic objects to the UI output module.

[0088] FIG. 7 is a diagram for explaining the operation of the image recognition unit and the position recognition unit in the guide robot according to the embodiment of the present invention.

[0089] A guide robot (eg, the guide robot 100 of FIG. 1) according to an embodiment may include an image recognition unit 730, a position recognition unit 735, a behavior control unit 740, a content recommendation unit 745, and a data management unit 750.

[0090] The image recognition unit 730 may include a sensor adapter module, a data transfer module, a character analysis module, and a target tracking module. For example, the sensor adapter module is a module that converts data acquired from the first image sensor 710 or the second image sensor 715 so that the data can be used by a module included in the image recognition unit 730. The character analysis module analyzes data (e.g., a target image) received from the sensor adapter module to search for an object. Specifically, the character analysis module recognizes at least one of a user's gender information, age information, face information, or any combination thereof from the target image.

[0091] The target tracking module generates tracking information for controlling the neck of the guide robot when recognizing a person in the target image. For example, the target tracking module generates data related to tracking of face information based on face information being recognized from the target image. Here, the tracking data refers to data for tracking and / or tracking face information recognized in the target image.

[0092] The data transmission module may be a module that publishes the processed and / or generated data (e.g., tracking data) to other components of the guide robot. For example, the data transmission module transmits at least one of the recognized information, the tracking data, or any combination thereof to the behavior control unit 740 and the content recommendation unit 745.

[0093] The position recognition unit 735 may include a sensor adapter module, a data fusion module, a path planning module, and a data transfer module. The sensor adapter module represents a module that converts data acquired from the first position sensor 720 or the second position sensor 725 so that the data can be used by modules included in the position recognition unit 735. The data fusion module represents a module that acquires direction information of the guide robot based on a map according to map information stored in the guide robot. The path planning module represents a module that generates a path for the guide robot to move based on the acquired information. The data transfer module may be a module that publishes the processed and / or generated data to other components of the guide robot.

[0094] Specifically, the data fusion module acquires a previous time point, which is a time point where the guide robot has moved and is close to the target time point, among previous time points, based on the target time point at which the position of the guide robot is recognized. The data fusion module acquires the direction of the guide robot at the target time point by combining the position of the guide robot at the previous time point and the position of the guide robot at the target time point with map information. The data transmission module transmits at least one of the position of the guide robot at the target time point, the direction of the guide robot at the target time point, or any combination thereof to the behavior control unit 740, the content recommendation unit 745, and the data management unit 750.

[0095] 8a to 8c are flowcharts illustrating an operation of the data management unit to update contents in the guide robot according to the embodiment of the present invention.

[0096] 8a to 8c, a guide robot (e.g., the guide robot 100 of FIG. 1) according to an embodiment updates content through a data management unit. Referring to FIG. 8a, the data management unit grasps or acquires the position of the guide robot in operation 801a. For example, the data management unit grasps or acquires the position of the guide robot based on the position acquired from the position recognition unit.

[0097] The data management unit determines whether the current position of the guide robot can be recognized in operation 803a. For example, based on the time when power is applied to the guide robot being less than a second predetermined threshold, the data management unit acquires the initial position and / or the current position of the guide robot based on the time when power is applied to the guide robot. However, the present invention is not limited thereto. For example, the data management unit acquires the direction of the guide robot in operation 803a.

[0098] In the operation 805a, the data management unit performs an operation to make the manager recognize the current position of the guide robot based on the fact that the current position of the guide robot could not be recognized.

[0099] In operation 807a, the data management unit updates the content based on the recognition of the current position of the guide robot. For example, the data management unit updates the content that is included in the map information and is different from the content version stored in the control server based on the fact that the guide robot has stopped at the initial position.

[0100] The data management unit determines whether the update was successful in operation 809a. For example, if the content update fails, the data management unit performs operation 807a again. Here, if the above-mentioned operation fails three times in operation 811a, the data management unit transmits an update failure notification to the behavior control unit. Specifically, if the content update fails, the guide robot performs a content output operation using the last version (or the latest version) of each content stored in the guide robot.

[0101] In operation 813a, the data management unit transmits a request to move the guide robot to a predetermined waiting position to the behavior control unit based on the completion of the update.

[0102] 8b, the data management unit checks whether the guide robot is in a standby state in operation 801b. For example, the data management unit checks whether the standby time of the guide robot exceeds 10 minutes in operation 803b. Specifically, the data management unit obtains the time during which the guide robot has been in an idle state based on the fact that the state of the guide robot is an idle state.

[0103] The data management unit checks the version of the content included in the map information based on the fact that the sustained time exceeds a predetermined first threshold (e.g., 10 minutes) in operation 805b. The data management unit updates the content that is included in the map information and is different from the content version stored in the control server in operations 807b to 809b. Operations 811b to 813b may be the same as those described in FIG. 8a. The data management unit transmits a request to the behavior control unit to move the guide robot to a predetermined standby position based on the fact that the data management unit has transmitted an update failure notification to the behavior control unit in operation 815b.

[0104] 8c, the data management unit checks whether the robot is in a charging state in operation 801c. Specifically, the data management unit obtains the charge amount of the guide robot based on the state of the guide robot being in a charging state in operation 803c, and performs the following operation based on the charge amount being equal to or greater than a third threshold value (e.g., 30%) determined in advance.

[0105] In operation 805c, the data management unit checks the version of the content included in the map information. In operations 807c to 809c, the data management unit updates the content that is included in the map information and is different from the version of the content stored in the control server. Operations 811c to 813c may be the same as those described in FIG. 8a.

[0106] In operation 815c, the data management unit transmits a request to the behavior control unit to maintain charging of the guide robot based on the update failure notification transmitted to the behavior control unit.

[0107] FIG. 9 is a flowchart illustrating a method for outputting a target content list in the guide robot according to an embodiment of the present invention.

[0108] A guide robot according to an embodiment (eg, the guide robot 100 in FIG. 1) answers the user's robot call in operation 910. The guide robot answers the user's robot call and performs the following operations.

[0109] The guide robot obtains a first subcontent list in operation 920. For example, the guide robot obtains the first subcontent list by confirming user characteristics and generating first feature data related thereto. The guide robot obtains a second subcontent list in operation 930. For example, the guide robot obtains the second subcontent list by confirming the position and direction of the guide robot and generating second feature data related thereto.

[0110] The guide robot obtains the target content list in operation 940. For example, the guide robot obtains the target content list by applying each of the first feature data and the second feature data to a content recommendation model. However, as described above, the method of obtaining the target content list is not limited thereto.

[0111] The guide robot outputs the target content list through the output unit in operation 950. The guide robot provides the user with the contents included in the target content list by outputting the target content list.

[0112] 10 is a diagram illustrating a control server according to an embodiment of the present invention. The control server 1000 according to an embodiment may include a processor 1010, a memory 1020 including an instruction code 1022, and a communication unit 1030. For example, the control server 1000 may be a server that controls a guide robot 1040 and transmits map information including contents to the guide robot 1040, or a server that transmits only contents.

[0113] The control server 1000 generates map information including a location where the guide robot 1040 is located and including contents that the guide robot 1040 can guide the user to. The control server 1000 extracts common contents that can be provided to the user in a first area (e.g., an area where the guide robot can move and other areas) based on the location where the guide robot 1040 is located from the contents stored in the control server 1000, and specialized contents that can be provided to the user in a second area (e.g., an area where the guide robot can move) based on the location where the guide robot 1040 is located. In response to a request from the guide robot 1040, the control server 1000 transmits the map information, the extracted common contents, and the extracted specialized contents to the guide robot 1040 via the communication unit 1030.

[0114] The processor 1010 may execute software and control at least one other component (e.g., a hardware or software component) coupled to the processor 1010. The processor 1010 may also perform various other data processing or calculations. For example, the processor 1010 may store map information, extracted common content, and extracted specialized content in the memory 1020.

[0115] For reference, the processor 1010 performs all operations performed by the control server 1000. Therefore, in this specification, for convenience of explanation, the operations performed by the control server 1000 are mainly described as operations performed by the processor 1010. Also, in this specification, for convenience of explanation, the processor 1010 is mainly described as one processor, but is not limited to this. For example, the control server 1000 may include at least one processor.

[0116] The memory 1020 temporarily and / or permanently stores various data and / or information required for operations performed by the control server 1000. For example, the memory 1020 stores map information, extracted common content, and extracted specialized content.

[0117] The communication unit 1030 supports communication between the control server 1000 and the guide robot 1040. For example, the communication unit 1030 may include one or more components for communicating between the control server 1000 and the guide robot 1040. For example, the communication unit 1030 may include a short range wireless communication unit, a microphone, and the like. In this case, the short range communication technology may include, but is not limited to, wireless LAN (Wi-Fi), Bluetooth (registered trademark), ZigBee, WFD (Wi-Fi Direct), UWB (ultra-wideband), infrared communication (IrDA, infrared Data Association), BLE (Bluetooth Low Energy), and NFC (Near Field Communication).

[0118] FIG. 11 is a diagram illustrating a method for transmitting map information and a content list to a guide robot in a control server according to an embodiment of the present invention.

[0119] According to an embodiment, the guide robot 1100 may include a data management unit that requests map information and a content list from the control server 1110. For example, the data management unit requests the map information and the content list from a first control server included in the control server 1110. The first control server responds to the request from the data management unit, checks the version of the requested map information, and checks the set content list. Specifically, the map information means information on a map that the guide robot 1100 outputs to a user along with the target content list. More specifically, the map information is information on a map of an area in which the guide robot 1100 can move and is stored as a file. Also, the content list may include information on the content stored in the guide robot 1100.

[0120] According to an embodiment, the control server 1110 may include a first control server and a second control server. The first control server and the second control server perform the same operations as the first control server (e.g., the first control server 510 in FIG. 5) and the second control server (e.g., the second control server 520 in FIG. 5) as described above in FIG. 5.

[0121] The first control server requests a content list from the second control server based on checking the version of the requested map information and the set content list. The second control server responds to the request from the first control server and transmits the content list to the first control server. The first control server transmits the content list to the guide robot 1100 based on receiving the content list from the second control server. For example, the first control server transmits a guide map in which common content and specialized content extracted from the content stored in the control server 1110 are displayed as predetermined areas based on the map information, and a content list related to the guide map, to the data management unit.

[0122] FIG. 12 is a diagram showing a guide map generated based on map information in the control server according to one embodiment of the present invention.

[0123] According to an embodiment, a control server (for example, the control server 160 in FIG. 1) generates a guide map 1200 based on map information. The guide map 1200 is provided to a user by a guide robot (for example, the guide robot 100 in FIG. 1) together with a target content list.

[0124] The guide map 1200 may include a map on which the guide robot moves and contents provided by the guide robot. For example, the guide map 1200 may include a contents list, an area type, a map name, an area, and a map.

[0125] The content list may include a content list that can be provided to the guide robot in the control server. Referring to Fig. 12, the content list may include a car explain content, a facility explain content, a repair information content, and a test drive content.

[0126] The area type may include a type in which the content included in the above-mentioned content list can be displayed on the guide map 1200. For example, the area type may include a rectangle type, a circle type, and a polygon type. For reference, in this specification, for convenience of explanation, the area type displayed on the guide map 1200 is mainly described as a rectangle type.

[0127] The map name indicates the name of the guide map 1200, more specifically, the file name. For example, the map name may be expressed as "Incheon Airport Flagship Store." That is, the guide map 1200 shown in FIG 12 may include a movable area where the guide robot can provide content at the flagship store of Incheon Airport among the airports.

[0128] The guide map 1200 is updated when the guide robot performs an initialization operation or is in an idle state. Also, the guide map 1200 is converted into an XML (Extensible Markup Language) or JSON (JavaScript Object Notation) format and transmitted to the guide robot. This will be described in detail later with reference to FIG. 13.

[0129] FIG. 13 is a diagram for explaining the source code of the guide map according to an embodiment of the present invention.

[0130] According to one embodiment, a control server (e.g., the control server 160 in FIG. 1) generates source code 1300 for a guide map. For example, the source code 1300 may include a name of the guide map (e.g., mapName), an area of ​​the guide map (e.g., mapArea), an action type (e.g., actionType), and information about the area (e.g., area).

[0131] The name of the guide map may be a name determined by the user and / or the administrator. The area of ​​the guide map may include an area where content is provided to the user on the guide map. Specifically, referring to FIG. 13, the area of ​​the guide map may include an area where vehicle description content is provided to the user and an area where facility guide is provided to the user.

[0132] The action type indicates content that can be provided to a user by the guide robot. In this regard, the area information may include information about an area where content is provided on the guide map according to the action type. Specifically, the area information may include a center point including an x-coordinate value based on the horizontal axis of the guide map and a y-coordinate value based on the vertical axis of the guide map, and may include a radius (e.g., radius) based on the center point.

[0133] FIG. 14 is a diagram illustrating a computing system relating to a guide robot, a control server, and a guide robot control method according to an embodiment of the present invention.

[0134] Referring to FIG. 14, a computing system 1400 relating to a guide robot, a control server, and a guide robot control method may include at least one processor 1410, a memory 1420, a user interface input device 1460, a user interface output device 1470, storage 1440, and a network interface 1450, all connected via a bus 1430.

[0135] The processor 1410 may be a central processing unit (CPU) or a semiconductor device that executes operations on instructions stored in the memory 1420 and / or the storage 1440. The memory 1420 and the storage 1440 may include various types of volatile or non-volatile storage media. For example, the memory 1420 may include a read only memory (ROM) 1421 and a random access memory (RAM) 1422.

[0136] Thus, the steps of a method or algorithm described in the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by the processor 1410, or in a combination of the two. The software modules may reside in a storage medium (i.e., memory 1420 and / or storage 1440) such as RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, etc.

[0137] An exemplary storage medium may be coupled to the processor 1410 such that the processor 1410 can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor 1410. The processor and the storage medium may reside in an application specific integrated circuit (ASIC). The ASIC may reside in a user terminal. Alternatively, the processor and the storage medium may reside as discrete components in a user terminal.

[0138] The above description is merely an illustrative example of the technical concept of the present invention, and various modifications and variations can be made by those having ordinary knowledge in the technical field to which the present invention pertains without departing from the essential characteristics of the present invention.

[0139] The embodiments described above may be implemented using hardware components, software components, and / or a combination of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as a processor, controller, arithmetic logic unit (ALU), digital signal processor, microcomputer, field programmable gate array (FPGA), programmable logic unit (PLU), microprocessor, or other device capable of executing and responding to instructions. The processing device executes an operating system (OS) and software applications that run on the operating system.

[0140] A processing device may also access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a processing device may be described as being a single device, but those skilled in the art will appreciate that a processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing device may include multiple processors or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0141] The software may include a computer program, code, instructions, or a combination of one or more of these, and may configure or instruct the processing device to operate as desired, either individually or collectively. The software and / or data may be permanently or temporarily embodied in some type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed over network-coupled computing systems and stored or executed in a distributed manner. The software and data may be stored on a computer-readable recording medium.

[0142] The method according to the embodiment may be embodied in the form of program instructions that may be executed by various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, and the like, alone or in combination, and the program instructions recorded on the medium may be those specifically designed and constructed for the embodiment, or may be those known and available to those skilled in the art of computer software. Examples of computer readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0143] The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

[0144] Although the embodiments have been described above with reference to the drawings, those skilled in the art may apply various technical modifications and variations based thereon. For example, the described techniques may be performed in a different order than described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than described, or may be counterbalanced or replaced by other components or equivalents, and still achieve suitable results.

[0145] Therefore, other implementations, embodiments, and equivalents of the claims are intended to be within the scope of the following claims.

[0146] Therefore, the embodiments disclosed in the present invention are for illustration purposes only and are not intended to limit the technical idea of ​​the present invention, and the scope of the technical idea of ​​the present invention is not limited by such embodiments. The scope of protection of the present invention should be interpreted according to the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention. [Explanation of symbols]

[0147] 100 Guide Robot 110 Image Recognition Unit 120 Location recognition unit 130 Behavior control unit 140 Content Recommendation Department 150 Communications Department 160 Control Server 301 Guide Robot 303 Input section including touch sensor and microphone 305 Sensors including image sensors and position sensors 307 Control Unit 309 UI input section 311 Location recognition unit 313 Image Recognition Department 315 Behavior Control Unit 317 Content Recommendation Department 319 Data Management Department 321 Output section 323 Control Server 400 Guide Robot 410 UI input section 420 Image Recognition Unit 430 Location recognition unit 440 Behavior Control Unit 450 Content Recommendation Department 460 Data Management Department 470 Output section 480 Control Server 500 Data Management Department 510 First Control Server 520 Second Control Server 610 UI input section 620 Image Recognition Unit 630 Location recognition unit 640 Behavior Control Unit 650 Content Recommendation Department 660 Output section 710 1st image sensor 715 2nd image sensor 720 1st position sensor 725 2nd position sensor 730 Image Recognition Unit 735 Location recognition unit 740 Behavior Control Unit 745 Content Recommendation Department 750 Data Management Department 1000 Control Server 1010 Processor 1020 Memory 1022 Command word 1030 Communications Department 1040 Guide Robot 1110 Control Server 1200 Guide Map 1400 Computer Systems 1410 processor 1420 Memory 1421 ROM 1422 RAM 1430 Bus 1440 Storage 1450 Network Interface 1460 User Interface Input Device 1470 User interface output device

Claims

1. A guide robot, A position recognition unit that recognizes a position of the guide robot; an image recognition unit for recognizing a user based on a target image acquired from an image sensor; a behavior control unit that transmits a request for generating a target content list reflecting the location of the guide robot and the characteristics of the user to a content recommendation unit in response to a call from the user, and outputs the target content list through an output unit of the guide robot based on the target content list received from the content recommendation unit; generating first characteristic data by combining the location of the guide robot with map information, and generating second characteristic data related to the characteristics of the user from the target image; and a content recommendation unit that transmits the target content list obtained by applying each of the first feature data and the second feature data to a content recommendation model to the behavior control unit.

2. The position recognition unit is Based on the target time point for recognizing the position of the guide robot, a previous time point is obtained, which is a time point where the guide robot has moved and is close to the target time point, among previous times; Acquire a direction of the guide robot at the target time by combining the position of the guide robot at the preceding time with the position of the guide robot at the target time with the map information; The guide robot according to claim 1 , further comprising: transmitting to the content recommendation unit at least one of a position of the guide robot at the target time, a direction of the guide robot at the target time, or a combination thereof.

3. The content recommendation unit includes: Generate the first feature data including at least one of a position of the guide robot at the target time, a direction of the guide robot at the target time, or any combination thereof; sorting at least one content included in the map information in order of proximity to the position of the guide robot included in the first characteristic data in a straight-line distance; applying the aligned content and the first feature data to the content recommendation model to obtain a first sub-content list including content related to a direction opposite to a direction of the guide robot; The first subcontent list includes: The guide robot according to claim 2, characterized in that the target content is included in the target content list.

4. The image recognition unit is Recognizing at least one of the user's gender information, age information, face information, or any combination thereof from the target image; generating information regarding tracking of the facial information based on the facial information being recognized from the target image; transmitting at least one of the recognized information, the tracking information, or any combination thereof to the content recommendation unit; The content recommendation unit includes: generating the second feature data including at least one of the recognized information, the tracking information, or any combination thereof; applying the second feature data to the content recommendation model to obtain a second sub-content list including content reflecting the user's characteristics among at least one content included in the map information; The second subcontent list includes: The guide robot according to claim 1, characterized in that the target content list includes the target content.

5. The map information further includes a data management unit that updates at least one content included in the map information, The data management unit Transmitting each version of at least one of the contents included in the map information to a control server; updating at least one of the versions of the content included in the map information, the version of the content being different from the version of the content stored in the control server; 2. The guide robot according to claim 1, further comprising: a controller configured to control a content update process for updating a content version included in the map information and different from the content version stored in the control server, the controller transmitting an update failure notification to the behavior control unit when an update of the content version included in the map information and different from the content version stored in the control server fails a predetermined number of times.

6. The data management unit obtaining an initial position of the guide robot based on a time when power is applied to the guide robot based on the time when power is applied to the guide robot being less than a first predetermined threshold value; Based on the fact that the guide robot has stopped at the initial position, updating the content that is included in the map information and is different from the content version stored in the control server; The guide robot according to claim 5, further comprising: a request to move the guide robot to a predetermined waiting position, based on the update failure notification being transmitted to the behavior control unit.

7. The data management unit Based on the state of the guide robot being an idle state, a time during which the idle state of the guide robot continues is obtained; updating content that is included in the map information and that is different from a content version stored in the control server based on the sustained time exceeding a second predetermined threshold; The guide robot according to claim 5, further comprising: a request to move the guide robot to a predetermined waiting position, based on the update failure notification being transmitted to the behavior control unit.

8. The data management unit Obtaining a charge amount of the guide robot based on the state of the guide robot being a charge state; updating content that is included in the map information and that is different from a content version stored in the control server based on the fact that the charged amount is equal to or greater than a third threshold value determined in advance; The guide robot of claim 5 , further comprising: a request to maintain charging of the guide robot transmitted to the behavior control unit based on the update failure notification transmitted to the behavior control unit.

9. The content recommendation unit includes:

2. The guide robot of claim 1, wherein, upon receiving resources related to the target content list from a control server, data related to a graphic object in which each of the contents included in the target content list is displayed as a predetermined area is transmitted to the behavior control unit.

10. A control server, a memory storing computer-executable instructions; at least one processor that accesses the memory and executes the instructions; A communication unit that communicates with the guide robot, The at least one processor Generate map information including a location where the guide robot is located and including content that the guide robot can guide to a user; extracting common content that can be provided to the user in a first area based on a location where the guide robot is located from the content stored in the control server, and extracting specialized content that can be provided to the user in a second area based on a location where the guide robot is located; a control server for transmitting the map information, the extracted common content, and the extracted specialized content to the guide robot in response to a request from the guide robot;

11. The at least one processor generating a guide map on which the extracted content is displayed as a predetermined area based on the map information; The control server of claim 10, further comprising: a control server configured to transmit the guide map to the guide robot in response to a resource request from the content recommendation unit of the guide robot.

12. An action of recognizing the position of the guide robot; Recognizing a user based on a target image acquired from an image sensor; transmitting a request for generating a target content list reflecting the location of the guide robot and the characteristics of the user to a content recommendation unit in response to a call from the user, and outputting the target content list through an output unit of the guide robot based on receiving the target content list from the content recommendation unit; generating first feature data by combining the location of the guide robot with map information, and generating second feature data related to the user's characteristics from the target image; and transmitting the target content list obtained by applying each of the first feature data and the second feature data to a content recommendation model to a behavior control unit.

13. The operation of recognizing the position of the guide robot includes: an operation of acquiring a previous time point, which is a time point where the guide robot has moved and is close to the target time point, among previous times based on the target time point for recognizing the position of the guide robot; acquiring a direction of the guide robot at the target time point by combining a position of the guide robot at the preceding time point and a position of the guide robot at the target time point with the map information; and transmitting at least one of a position of the guide robot at the target time, a direction of the guide robot at the target time, or any combination thereof to the content recommendation unit.

14. An operation of generating the first feature data including at least one of a position of the guide robot at the target time point, a direction of the guide robot at the target time point, or any combination thereof; sorting at least one content included in the map information in order of proximity to the position of the guide robot included in the first characteristic data; 14. The method of claim 13, further comprising: applying the aligned contents and the first feature data to the content recommendation model to include contents related to a direction opposite to a direction of the guide robot, and acquiring a first sub-content list to be included in the target content list.

15. The operation of recognizing the user includes: Recognizing at least one of the user's gender information, age information, face information, or any combination thereof from the target image; generating information regarding tracking of the face information based on the face information being recognized from the target image; transmitting at least one of the recognized information, the tracking information, or any combination thereof to the content recommendation unit; generating the second feature data, the second feature data including at least one of the recognized information, the tracking information, or any combination thereof; and applying the second feature data to the content recommendation model to obtain a second sub-content list to be included in the target content list, the second sub-content list including content reflecting the user's characteristics among at least one content included in the map information.

16. transmitting at least one version of each of the at least one content included in the map information to a control server; updating at least one of the versions of the content included in the map information, the version of the content being different from the version of the content stored in the control server; 13. The method of claim 12, further comprising: transmitting an update failure notification to the behavior control unit when an update of content, which is included in the map information and is different from the content version stored in the control server, has failed a predetermined number of times.

17. The update operation includes: obtaining an initial position of the guide robot based on a time when power is applied to the guide robot based on the time when power is applied to the guide robot being less than a first predetermined threshold; an operation of updating the content, which is included in the map information and is different from the content version stored in the control server, based on the fact that the guide robot has stopped at the initial position; and transmitting, to the behavior control unit, a request to move the guide robot to a predetermined waiting position based on the transmission of the update failure notification to the behavior control unit.

18. The update operation includes: acquiring a duration of the idle state of the guide robot based on the state of the guide robot being an idle state; updating a content version included in the map information and different from a content version stored in the control server based on the duration exceeding a second predetermined threshold; and transmitting, to the behavior control unit, a request to move the guide robot to a predetermined waiting position based on the transmission of the update failure notification to the behavior control unit.

19. The update operation includes: obtaining a charge amount of the guide robot based on the state of the guide robot being a charge state; an operation of updating content that is included in the map information and that differs from a content version stored in the control server, based on the fact that the charged amount is equal to or greater than a third threshold value determined in advance; and transmitting a request to the behavior control unit to maintain charging of the guide robot based on the transmission of the update failure notification to the behavior control unit.

20. generating map information including a location where the guide robot is located and including content that the guide robot can guide a user to; extracting common content that can be provided to the user in a first area based on a location where the guide robot is located from among the contents stored in the control server, and extracting specialized content that can be provided to the user in a second area based on a location where the guide robot is located; The method of claim 12, further comprising: transmitting the map information, the extracted common content, and the extracted specialized content to the guide robot in response to a request from the guide robot.

Images