Guiding robot and operation method of guiding robot

Abstract

The invention discloses a guide robot and an operation method of the guide robot. When activating a monitoring function, the guide robot of the present invention not only travels in a monitoring area with a specified travel path, but can adaptively change the travel path for monitoring and monitor the surroundings on the basis of information relating to the state of the robot or the state of a travel space. In addition, if an event requiring additional monitoring is identified, the system leaves the monitoring area and performs additional monitoring, and can actively perform appropriate corresponding actions according to the result of additional monitoring.

Description

Technical Field

[0001] The present invention relates to a guiding robot and a method for operating the guiding robot, and more particularly to a guiding robot capable of autonomously driving and performing a monitoring function at the same time and a method for operating the guiding robot. Background Art

[0002] In recent years, people have shown increasing interest in guidance robots that provide various guidance services to users. Guidance robots can travel autonomously without user operation and provide various guidance services to users through interactive functions such as voice dialogue functions or touch screens.

[0003] For example, normally, the guidance robot mainly performs the guidance function during the daytime when there are many tourists, and mainly performs the monitoring function during the nighttime when there are no tourists. However, in order to strengthen security, the monitoring function is also used during the daytime when necessary.

[0004] The monitoring function of the guide robot is generally a function of monitoring by a surveillance camera while the guide robot travels along a specified travel path in a specified space. Specifically, after the positions that the guide robot needs to pass are listed in sequence, the guide robot performs monitoring while passing through the positions.

[0005] That is, the monitoring function of the existing guide robot is at the level of recording CCTV (Closed Circuit Television) with a camera while repeatedly driving along a designated driving path. In fact, this is almost the same as the operation level of a fixed CCTV.

[0006] In addition, the guide robot uses a rechargeable battery to travel and operate, so the guide robot repeatedly travels along a designated travel path and moves to a charging station for charging the battery when the remaining battery power is insufficient. The guide robot consumes the most battery power during travel, so the uniform travel monitoring as described above consumes the battery power faster, resulting in a significant reduction in the operable time of the monitoring function.

[0007] Therefore, more efficient monitoring functions can be achieved by reflecting the characteristics of autonomous and increasingly intelligent guidance robots. Summary of the invention

[0008] Problems to be solved by the invention

[0009] To this end, according to some embodiments of the present invention, it is an object to provide a guide robot and a method of moving the guide robot that can achieve efficient monitoring that conforms to the current state of the guide robot and the characteristics of the driving space by reflecting the characteristics of the guide robot that can drive autonomously and is increasingly intelligent.

[0010] In addition, according to some embodiments of the present invention, it is an object to provide a guide robot and a method for operating the guide robot that can perform monitoring while adaptively changing a travel path in consideration of the current state of the guide robot and the characteristics of a travel space.

[0011] In addition, according to some embodiments of the present invention, an object is to provide a guidance robot and a guidance robot operation method that can autonomously determine and execute appropriate corresponding actions based on monitoring results.

[0012] In addition, according to some embodiments of the present invention, the purpose is to provide a guiding robot and a guiding robot action method that can leave a designated monitoring area as needed and continuously perform abnormality detection, and can perform actions corresponding to the abnormality detection results differently depending on the situation.

[0013] Technical solutions to the problem

[0014] When the monitoring function is activated, the guide robot of the present invention not only travels in the monitoring area along a designated driving path, but can also monitor the surroundings while adaptively changing the driving path according to the state of the robot or the state of the driving space.

[0015] In addition, during monitoring in the monitoring area, if an event requiring additional monitoring is identified, the monitoring area will be left to perform additional monitoring, and appropriate corresponding actions can be actively performed based on the additional monitoring results.

[0016] Specifically, the guide robot of the embodiment of the present invention includes: a camera, which, when activated, photographs the surroundings of the guide robot; a driving unit, which moves the guide robot; a detection unit, which collects status information of the guide robot; and a processor, which is electrically connected to the camera, the driving unit, and the detection unit. In addition, as the monitoring mode is executed, the processor can activate the camera, and in the monitoring mode, the guide robot can receive the status information of the guide robot while driving in the monitoring area, and can determine the driving path of the monitoring area based on the received status information of the guide robot.

[0017] In an embodiment, the processor may determine the driving path based on the field of view angle range of the camera to monitor the entire surveillance area.

[0018] In an embodiment, the status information of the guide robot may include at least one of the remaining battery power of the guide robot, the estimated time required to monitor the entire monitoring area, and the presence of other guide robots that can cooperate.

[0019] In an embodiment, the processor can execute a power saving mode according to the remaining battery power of the guide robot. As the power saving mode is executed, the guide robot can be moved to a specific position within the monitoring area and monitor the monitoring area by controlling the traveling part. At this time, the specific position can be changed according to the status information collected in the monitoring area.

[0020] In an embodiment, during driving in the monitoring area, the processor may identify the occurrence of a preset monitoring event, execute an action corresponding to the identified monitoring event, and then monitor the monitoring area.

[0021] In an embodiment, the pre-set monitoring event may be a situation context related to at least one of visitor status monitoring, visitor epidemic prevention monitoring, protected area monitoring, and status monitoring within a building.

[0022] In an embodiment, the processor can calculate a POI position for performing an action corresponding to the identified monitoring event based on the location of the occurrence of the situation scenario, and can set the next driving path based on the calculated POI position and the current position of the guide robot.

[0023] In an embodiment, a touch screen displaying information related to the identified monitoring event may be further included, the processor may be electrically connected to the touch screen, and may request a feedback response of an action corresponding to the identified monitoring event through the touch screen.

[0024] In an embodiment, the detection unit can collect status information of the driving space, and the processor can, in the monitoring mode, determine the next driving direction based on the status information of the driving space collected during driving in the monitoring area, and can confirm the monitoring object corresponding to the status information of the driving space collected while driving in the determined driving direction, and can perform corresponding actions related to the confirmed monitoring object.

[0025] In an embodiment, the state information of the driving space may be state data including at least one of temperature data, slip data, and inclination data of the driving space.

[0026] In an embodiment, the next travel direction of the guide robot may be determined as a direction associated with a position where an abnormality is detected, where an abnormality is detected in an analysis result of the state data of the travel space.

[0027] In an embodiment, the processor can set a candidate POI for confirming the monitored object based on the location of the anomaly detection, select one of the candidate POIs with a higher expected weight of anomaly detection as the search POI, travel at the search POI and accumulate status information of the driving space, and detect the next search POI based on the collection results.

[0028] In an embodiment, at least a portion of the search POI or a subsequent search POI may be located outside the monitoring area.

[0029] In an embodiment, a touch screen for displaying information related to the confirmed corresponding action related to the monitored object may also be included.

[0030] In an embodiment, the guiding robot may further include a communication unit. After the processor confirms that there is no feedback response regarding the displayed information within a preset time, the communication unit may transmit the confirmation result of the monitored object and the corresponding action related to the confirmed monitored object to a preset administrator terminal.

[0031] In addition, the action method of the guiding robot of an embodiment of the present invention may include: the step of activating the camera of the guiding robot and executing the monitoring mode; the step of acquiring the status information of the guiding robot while driving in the monitoring area in the monitoring mode; and the step of determining the driving path of the monitoring area as different ones based on the acquired status information of the guiding robot.

[0032] Effects of the Invention

[0033] According to some embodiments of the present invention, the guide robot and the operation method thereof can take into account the field of view of the monitoring camera and can actively set the driving path of the monitoring area or change the existing driving path based on various status information such as the remaining battery power, the estimated time required to monitor the entire monitoring area, and the existence of other guide robots that can cooperate. In this way, more effective and intelligent monitoring can be performed instead of uniform monitoring.

[0034] In addition, the guiding robot and its action method according to some embodiments of the present invention can realize more detailed or specialized monitoring such as epidemic prevention and protection area monitoring, and when problems or events occur, they can track related objects and perform appropriate corresponding actions.

[0035] In addition, as an appropriate response action, a feedback response can be requested, and if there is no response to the request, the administrator can be notified. This makes it possible to achieve more appropriate and detailed abnormality detection and response actions according to the situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 : is a diagram showing an example of a guide robot related to the present invention.

[0037] Figure 2 : is a block diagram showing an example configuration of a guide robot related to the present invention.

[0038] Figure 3 This is a diagram for explaining the monitoring function of a general guidance robot.

[0039] Figure 4 This is a representative flowchart for explaining the operation method of the guidance robot related to the present invention.

[0040] Figure 5 It is a diagram showing that a monitoring function is performed in consideration of the field angle range of a camera in a guide robot related to the present invention.

[0041] Figure 6a , Figure 6b as well as Figure 6c Various examples are shown in which, in the guide robot according to the present invention, different travel routes are set for a monitoring area based on different states of the guide robot.

[0042] Figure 7 , Figure 8a , Figure 8b , Fig. 9 as well as Fig.10 This is a diagram for explaining an example in which the guide robot according to the present invention recognizes the occurrence of a monitoring event and performs a corresponding action while traveling in a monitoring area.

[0043] Fig.11 This is another flowchart for explaining the operation method of the guiding robot related to the present invention.

[0044] Fig.12 , Fig.13 as well as Fig.14 The diagram is used to explain a method for a guidance robot related to the present invention to detect an abnormality in a cooling or heating state and confirm a monitoring object, as well as corresponding actions related thereto.

[0045] Fig.15a , Fig.15b as well as Fig.15c This is a diagram for explaining an example of a method in which a guidance robot according to the present invention detects an abnormal inclination of a floor of a building and confirms a management status. DETAILED DESCRIPTION

[0046] Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings. Regardless of the figure numbers, the same or similar constituent elements will be marked with the same figure numbers, and repeated descriptions thereof will be omitted. The suffixes "module" and "unit" of the constituent elements used in the following description are only given or mixed in consideration of the ease of writing of the description, and they themselves do not have meanings or functions that distinguish each other. In addition, when describing the embodiments disclosed in this specification, if it is determined that the specific description of the relevant known technology may confuse the subject matter of the embodiments disclosed in this specification, its detailed description will be omitted. In addition, the accompanying drawings are only used to make the embodiments disclosed in this specification easy to understand, and it should be understood that the technical ideas disclosed in this specification are not limited by the accompanying drawings, but cover all changes, equivalents and substitutes included in the ideas and technical scope of the present invention.

[0047] Terms including ordinal numbers such as first, second, etc. may be used to describe various constituent elements, but the constituent elements are not limited by the terms. The terms are only used to distinguish one constituent element from other constituent elements.

[0048] When it is mentioned that a certain component is “connected” or “linked” to another component, it should be understood that it can be directly connected or linked to another component, but other components may also exist between them. On the contrary, when it is mentioned that a certain component is “directly connected” or “directly linked” to another component, it should be understood that there are no other components between them.

[0049] Unless clearly indicated otherwise in the context, an expression in the singular includes an expression in the plural.

[0050] In the present application, it should be understood that terms such as “including” or “having” are intended to indicate the existence of the features, numbers, steps, actions, constituent elements, parts or combinations thereof recorded in the specification, and do not preclude the possibility of the existence or increase of one or more other features or numbers, steps, actions, constituent elements, parts or combinations thereof.

[0051] On the other hand, the "guidance robot" disclosed in this specification refers to a robot that can provide users with a variety of information such as welcome greetings, route guidance, product guidance, product retrieval, parking guidance, airport information, docent information, library guidance, etc. in various public places such as airports, shopping centers such as department stores, accommodation facilities such as hotels, art galleries, and cultural spaces such as libraries.

[0052] In addition, the "guidance robot" disclosed in this specification can perform autonomous driving by itself in order to guide the user to a route, a specific place, etc.

[0053] In addition, when the “guide robot” disclosed in this specification travels in a designated space, objects moving in the designated space may be referred to as visitors, users, obstacles, and the like.

[0054] In addition, the "guidance robot" disclosed in this specification may include various output mechanisms related to a touch screen, a sound output unit, an LED (light emitting diode), a tactile sensor, etc., in order to provide information or guidance to the user in various ways (visually, auditorily, and tactilely).

[0055] Figure 1 is a diagram showing an example of a guide robot 100 related to the present invention.

[0056] Reference Figure 1 The guide robot 100 of the present invention may include a head 102, a camera 121, a speaker 152, a voice recognition unit (not shown), a display 151, and a travel unit 130. However, depending on the situation, the guide robot 100 of the present invention may be implemented by removing part of the mechanisms disclosed in the present invention or including other mechanisms.

[0057] The appearance of the guide robot 100 of the present invention is roughly formed by a head 102, an upper module including a display 151, and a lower module including a travel portion 130. The upper module and the lower module may be provided to be attachable to and detachable from each other.

[0058] The upper module provides a user interface that can be changed according to the service environment. The lower module provides a driving function for guiding the movement of the robot body.

[0059] The upper module may also form a main body and may be divided into a main body provided with a display 151 and a head 102 provided with a camera 121, etc. However, depending on circumstances, a camera may be provided in the main body or a touch screen may be configured in the head 102.

[0060] The camera 121 may be disposed on one side of the housing of the head 102 or on one side of the housing of the main body. In addition, a plurality of cameras 121 may be provided. In this case, one camera may be disposed on the front of the main body and face forward, and another camera may be disposed on the side or back and face sideways / backward. Thus, a 360-degree field of view may be formed.

[0061] In the case where a plurality of cameras 121 are provided, the first camera may include, for example, a 3D stereo camera. The 3D stereo camera may perform functions such as obstacle sensing, user facial recognition, and stereo image acquisition. The guiding robot 100 may use the first camera to sense and avoid obstacles in its own moving direction, and may recognize the user and perform various control actions. In addition, the second camera may include, for example, a Simultaneous Localization And Mapping (SLAM) camera. The SLAM camera tracks the current position of the camera by matching specific points, and performs based on this.

[0062] In addition, the camera 121 can identify objects within the field of view, and can perform the function of taking photos and videos. In this regard, the camera 121 may include at least one of a camera sensor (e.g., a charge-coupled device (CCD), a complementary metal oxide semiconductor (CMOS), etc.), a light sensor (or an image sensor), and a laser sensor. The camera 121 and the laser sensor can be combined with each other and sense the touch of a sensing object of a three-dimensional stereoscopic image. The light sensor can be stacked on the display device, and such a light sensor is formed to scan the movement of a sensing object close to the touch screen. More specifically, the light sensor scans the content placed on the light sensor by installing photodiodes (Photo Diode) and transistors (Transistor; TR) in rows / columns and using an electrical signal that changes according to the amount of light applied to the photodiode (Photo Diode). That is, the light sensor can calculate the coordinates of the sensing object based on the amount of light change, thereby obtaining the position information of the sensing object.

[0063] The sound output unit 152 can perform a function of notifying the information provided to the user in a voice manner, and can be, for example, a speaker. Specifically, the sound output unit 152 outputs a response or a search result corresponding to the user's voice received by the sound receiving unit 122 and the voice recognition unit (not shown) provided in the guide robot 100 in a voice manner. Such a sound output unit 152 can be provided on the head 102 or the outer peripheral surface of the main body provided with the display 151. In addition, the sound output unit 152 can output voice information related to the screen displayed on the display 151 (for example, a menu screen, an advertisement screen, etc.).

[0064] The sound receiving unit 122 can perform a function of receiving the user's voice, etc., for example, it can be a microphone. The sound receiving unit 122 can process an external sound signal into electrical voice data, and can implement various noise removal algorithms for removing noise generated during the process of receiving the external sound signal.

[0065] The display 151 may be located in a longitudinal direction of the main body and may display a picture in order to provide visual information, for example, guide information. In addition, the display 151 may include a display module, a touch sensor, and a pressure sensor.

[0066] The display 151 may be implemented, for example, to be combined with the movement guide mechanism and to open and close the interior of the main body. In addition, the display 151 may also be implemented, for example, to be fastened and fixed to the main body by using a fixing member.

[0067] In addition, although not shown in detail, considering the situation that the guide robot 100 moves in a straight line along a set path in order to guide the user, the display 151 can be set at the rear of the head 102, or can be set at the rear in addition to the front. Alternatively, before moving in a straight line along the set path, the head 102 can be rotated 180 degrees to change the appearance, as if the display 151 is located at the rear.

[0068] In this case, the display 151 performs a function of displaying visual information related to the currently provided service (eg, route guide information, question information). The user can watch the display 151 provided at the rear of the guide robot 100 while following the movement of the guide robot 100 .

[0069] In addition, the display 151 can be respectively arranged on the front and back sides of the main body. In this case, the first display arranged on the front of the main body and the second display arranged on the back of the main body can display different pictures (for example, the first display displays the picture of interaction with the user, and the second display displays the advertising picture, etc.). In addition, a display unit for outputting various expression changes of the guiding robot can be set on the front of the head 102.

[0070] The travel unit 130 guides the movement and rotation of the main body of the robot 100. To this end, the travel unit 130 may include a plurality of wheels and a drive motor. The driving of the travel unit 130 may be controlled according to the control instruction received by the processor, and a notification may be provided through the LED output mechanism 153 before and after the driving.

[0071] Figure 2 is a block diagram showing an exemplary detailed configuration of a guide robot related to the present invention.

[0072] The guide robot 100 of the present invention may include a communication unit 110 , an input unit 120 , a travel unit 130 , a detection unit 140 , an output unit 150 , a memory 170 , a processor 180 , a power supply unit 190 , and the like. Figure 2 The components shown in the figure are not essential for realizing the guiding robot, and the components of the guiding robot described in this specification may be more or less than the components listed above.

[0073] The communication unit 110 may include at least one module capable of wirelessly communicating between the guide robot 100 and an external server, such as an artificial intelligence server, or an external terminal. In addition, the communication unit 110 may include at least one module for connecting the guide robot 100 to at least one network.

[0074] The communication unit 110 can communicate with the artificial intelligence server, etc. by using wireless Internet communication technologies such as Wireless LAN (WLAN), Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, Digital Living Network Alliance (DLNA), Wireless Broadband (WiBro), World Interoperability for Microwave Access (WiMAX), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Long Term Evolution (LTE), LTE-Advanced, etc. In addition, the communication unit 110 can use Bluetooth (Bluetooth TM ), Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC) and other short-range communication technologies to communicate with external terminals.

[0075] The input unit 120 may include a camera 121 or an image input unit for image signal input, a sound receiving unit 122 for audio signal input, such as a microphone, and a user input unit for receiving information from a user (not shown, such as a touch key, a mechanical key, etc.). The signal data, voice data, and image data collected from the input unit 120 may be analyzed and processed into control instructions.

[0076] The travel unit 130 can guide the movement and rotation of the main body of the robot 100. To this end, the travel unit 130 may include a plurality of wheels and a drive motor. The driving of the travel unit 130 can be controlled according to the control instructions received by the processor 180, and before and after the driving, a notification can be provided through the light output unit 153 such as an LED.

[0077] The detection unit 140 may include at least one sensor for detecting at least one of information in the guide robot, information about the surrounding environment surrounding the guide robot, and user information. For example, the detection unit 140 may include at least one of a proximity sensor 141, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, a gravity sensor (G-sensor), a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor (IR sensor: infrared sensor), a fingerprint sensor, an ultrasonic sensor, an optical sensor (for example, a camera (refer to reference numeral 121)), a microphone, a battery gauge, an environmental sensor (for example, a barometer, a hygrometer, a thermometer, a radiation sensing sensor, a heat sensing sensor, a gas sensing sensor, etc.), and a chemical sensor (for example, an electronic nose, a health sensor, a biosensor, etc.). On the other hand, the guide robot disclosed in the present specification may combine and use information detected from at least two of such sensors. In addition, the detection unit 140 may include a driving-related sensor 142 for sensing obstacles, ground conditions, and the like.

[0078] Examples of the proximity sensor 141 include a light-transmitting photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high-frequency oscillation proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, an infrared proximity sensor, etc. In addition, the proximity sensor 141 may include at least one of a navigation camera, an ultrasonic sensor, a lidar, and a time-of-flight (ToF) sensor, thereby identifying the proximity and position of a sensing object (e.g., a user).

[0079] In addition, the detection unit 140 may include at least one sensor for collecting status information, for example, a camera, a 3D depth camera, a laser radar, a speed sensor, a distance sensor, an obstacle sensor, a proximity sensor 141, or two or more sensors.

[0080] The detection unit 140 may collect status information by using one or more sensors. Here, the collected status information may include, for example, whether there are tourists in the space, whether the tourists are approaching or moving away and their directions, the driving speed of the robot, and the characteristics of the driving space, etc. In addition, the status information may also include external characteristics (e.g., gender, age, etc.) of the approaching or moving away tourists.

[0081] The status information collected by the detection unit 140 may be provided to the processor 180 or stored in the memory 170 .

[0082] The processor 180 may adjust the volume of the sound outputted by the sound output unit 152 to be different according to the collected status information. More specifically, the processor 180 may subdivide the sound objects or purposes of the robot and set the adjustment range of the corresponding sound volumes to be different.

[0083] The processor 180 may select a sound emission mode to adjust the volume of the sound emission within a first setting range, or select a sound emission mode to adjust the volume of the sound emission within a second setting range different from the first setting range, based on the collected status information.

[0084] At this time, the maximum value of the first setting range may be a value greater than the maximum value of the second setting range. For example, the first setting range may be selected as a sounding mode for making sounds for multiple people ('public sounding mode'), and the second setting range may be selected as a sounding mode for making sounds for a specific individual close to the robot ('individual sounding mode').

[0085] When the display 151 is respectively provided on the front and back of the main body, the processor 180 can be controlled so that the front display and the back display display different contents (e.g., different advertisements). In this case, the processor 180 can control the sound output unit 152 to output the sound related to the display on the side with high advertising effect, important in safety, and high control priority according to the collected status information.

[0086] The output unit 150 is used to generate output related to vision, hearing or touch, and may include at least one of a display 151, a sound output unit 152, and a light output unit 153. The display 151 may form a layer structure or be integrated with a touch sensor to realize a touch screen. Such a touch screen may function as a user input unit that provides an input interface between the guiding robot 100 and the user, while providing an output interface.

[0087] The light output unit 153 outputs a signal for notifying the occurrence of an event to the guide robot 100 using light from a light source. For example, when a movement instruction is transmitted to the travel unit 130 of the guide robot 100, the signal for notifying the movement is outputted by the light output unit 153.

[0088] In order to perform actions related to the artificial intelligence technology of the guide robot, the processor 180 may include an AI learning unit 181 (not shown). The AI ​​learning unit 181 may be configured to receive, classify, store, and output information applied to data mining, data analysis, intelligent decision-making, and machine learning algorithms and technologies. The AI ​​learning unit 181 may include at least one memory unit, which is configured to store information received, detected, sensed, generated, pre-defined by the guide robot or information output by the guide robot in other ways, or configured to store data received, detected, sensed, generated, pre-defined, and output by other components, devices, and terminals.

[0089] In one embodiment, the AI ​​learning unit 181 may be integrated into the guide robot, or may include a memory. In one embodiment, the AI ​​learning unit 181 may be implemented by the memory 170. However, not limited thereto, the AI ​​learning unit 181 may be implemented in an external memory associated with the guide robot 100, or may be implemented by a memory included in a server capable of communicating with the guide robot 100. In another embodiment, the AI ​​learning unit 181 may be implemented by a memory maintained in a cloud computing environment, or may be implemented by other remote memories that can be accessed by the guide robot using a communication method such as a network.

[0090] In order to identify, index, classify, manipulate, store, retrieve and output data for supervised or unsupervised learning, data mining, predictive analysis and other machine learning techniques, the AI ​​learning unit 181 is generally implemented to store the data in at least one database. The information stored in the AI ​​learning unit 181 can be used by the processor 180 or a plurality of processors (processors) included in the guiding robot, which use at least one of different types of data analysis, machine learning algorithms and machine learning techniques.Examples of such algorithms and techniques are k-nearest neighbor systems, fuzzy logic (e.g., possibility theory), neural networks, Boltzmann machines, vector quantization, pulsed neural nets, support vector machines, maximum margin classifiers, hill-climbing, inductive logic systems, Bayesian networks, petri nets (e.g., finite state machines, mealy machines, Moore finite state machines), classifier trees (e.g., perceptron trees, support vector trees, Markov trees, decision tree forests, random forests), pandemonium models and systems. and systems), clustering, artificially intelligent planning, artificially intelligent forecasting, data fusion, sensor fusion, image fusion, reinforcement learning, augmented reality, pattern recognition, automated planning, etc.

[0091] The processor 180 can determine or predict executable actions to guide the robot based on information determined or generated using data analysis, machine learning algorithms, and machine learning techniques. To this end, the processor 180 can request, retrieve, receive, or use data from the AI ​​learning unit 181. The processor 180 can perform various functions for implementing knowledge-based systems, reasoning systems, and knowledge acquisition systems, and can perform various functions including systems for uncertain reasoning (e.g., fuzzy logic systems), adaptive systems, machine learning systems, artificial neural networks, etc.

[0092] In addition, the processor 180 may include a plurality of submodules that can process speech and natural language, such as an input / output (I / O) processing module, an environmental condition module, a speech-to-text (STT) processing module, a natural language processing module, a task flow processing module, and a service processing module. Each of the plurality of submodules may have access rights to at least one system, data, and model that guides the robot, or a subset or superset thereof. Here, each of the plurality of submodules may have access rights to a scheduling, vocabulary index, user data, task flow model, service model, and automatic speech recognition (ASR) system.

[0093] In some embodiments, based on the data in the AI ​​learning unit 181, the processor 180 can be configured to detect and sense the user's needs based on the contextual conditions or the user's intention expressed by the user input or natural language input. Based on the data analysis, machine learning algorithm and machine learning technology performed by the AI ​​learning unit 181, if the action of the guide robot is determined, the processor 180 controls the components of the guide robot in order to perform the determined action. The processor 180 can control the guide robot according to the control instruction to perform the determined action.

[0094] The memory 170 stores data supporting various functions of the guide robot 100. The memory 170 may store a plurality of application programs (executable programs or application software) driven in the guide robot 100 and data and instructions for guiding the actions of the robot 100. In addition, the memory 170 may store a variable call word for performing a voice dialogue function with a user.

[0095] The memory 170 may store status information and data collected by the guide robot 100 through the detection unit 140 , the input unit 120 , the communication unit 110 , etc. In addition, the status information and data stored in the memory 170 may be provided to the processor 180 .

[0096] The memory 170 may include, for example, at least one type of storage medium selected from a flash memory type, a hard disk type, a solid state disk type, a silicon disk drive type, a multimedia card micro type, a card type memory (e.g., a secure digital (SD) memory or an extreme digital-picture (XD) memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a disk, and an optical disk.

[0097] In addition to the actions related to the application, the processor 180 generally controls the overall actions of the robot 100. The processor 180 can process signals, data, information, etc. input or output through the above-described components, or drive the application stored in the memory 170, or control the travel unit 130, thereby providing or processing appropriate information or functions to the user.

[0098] Under the control of the processor 180, the power supply unit 190 receives external power and internal power to supply power to each component included in the guide robot 100. The power supply unit 190 may include a battery, which may be a built-in battery or a replaceable battery.

[0099] In order to realize the actions, controls, and control methods of the guide robot in various embodiments described below, at least a portion of the components may cooperate with each other and operate. In addition, the actions, controls, and control methods of the guide robot may be realized on the guide robot by driving at least one application program stored in the memory 170.

[0100] In addition, various embodiments disclosed below can be implemented in a recording medium that can be read by a computer or a device similar thereto by using, for example, software, hardware, or a combination thereof.

[0101] Hereinafter, a guidance robot capable of autonomous driving and making sounds by adjusting the volume and an operation method thereof according to an embodiment of the present invention will be described in detail.

[0102] first, Figure 3 This is a diagram for explaining the monitoring function of a general guidance robot. Figure 3 If the existing guiding robot 20 performs the monitoring function on the monitoring area 310 in the specified space 10, it will move in a manner of traveling along the specified direction (for example, counterclockwise) to the boundary of the set monitoring area 310 and performing camera monitoring.

[0103] Specifically, if the administrator sets the boundary vertices (1, 2, 3, 4) of the monitoring area 310 as points of interest (POI), the driving path 311 will be set to guide the robot to pass through the POI (1, 2, 3, 4) in sequence, and the monitoring function will be performed while repeatedly driving through the POI (1, 2, 3, 4).

[0104] This means that monitoring of a portion of the inner side 310′ of the monitoring area and a portion of the outer side of the monitoring area 310 will be excluded, thereby reducing the utility. In particular, when the size of the monitoring area 310 is larger and / or there is a restricted field of view (for example, a restricted field of view caused by a pillar, etc.), the utility is further reduced, and thus the monitoring area 310 cannot be properly monitored. In addition, the monitored area will also change with the direction of travel of the guide robot, so there is a problem that it is difficult for the administrator to consider all factors and generate a travel path in advance.

[0105] Therefore, in this specification, by reflecting the characteristics of a guide robot that can travel autonomously and gradually become intelligent, a guide robot is realized that can perform efficient monitoring that conforms to the current state of the guide robot and the characteristics of the driving space, and can adaptively change the driving path and perform monitoring.

[0106] Figure 4 is a representative flow chart for explaining the method of guiding the robot related to the present invention. Unless otherwise specified, Figure 4 Each step of the guide robot 100 is executed by the processor 180. In addition, when the guide robot 100 performs the monitoring function, the following steps may be repeatedly performed: Figure 4 In addition, although not shown in detail, the guiding robot 100 can Figure 4The process performs AI learning, updates and / or establishes a database based on the monitoring results of the monitored area.

[0107] Reference Figure 4 , first, the guide robot 100 may activate a camera and execute a monitoring mode (S10).

[0108] Here, the camera may include, in addition to a general camera, a camera for special monitoring such as a depth camera, a two-dimensional camera, a three-dimensional camera, a thermal imaging camera, etc. In addition, as the monitoring mode of the guiding robot is executed, the camera may be activated to perform a permanent recording function.

[0109] As described above, if the monitoring mode is executed, the guide robot receives state information of the guide robot collected while traveling in the monitoring area ( S20 ).

[0110] Here, the status information of the guide robot refers to the status data of the guide robot that affects the guide robot's execution of the monitoring function, for example, it may include at least one of the remaining battery power of the guide robot, the estimated time required to monitor the entire monitoring area, and whether there is another guide robot that can cooperate.

[0111] Such status information of the guiding robot can be collected or acquired through, for example, status information-related sensors 143 (for example, a battery remaining sensing sensor, an operation time measuring sensor, etc.) of the detection unit 140 of the guiding robot 100, the communication unit 110, the camera 121, and the input unit 120 such as the sound receiving unit 122.

[0112] The collected or acquired state information of the guiding robot may be stored in the memory 170 , or transmitted to the outside through the communication unit 110 , or perform related actions, or transmitted to the processor 180 for AI learning and / or visualized through the display 151 .

[0113] Then, the guide robot 100 may change the driving path of the monitoring area based on the received or acquired state information of the guide robot ( S30 ).

[0114] Here, changing (or setting to be different) the driving path includes both the case of adaptively changing the existing driving path and the case of resetting the driving path based on the state information of the guide robot.

[0115] In the former case, the adaptive change of the existing driving path can be recorded. In addition, if the state data of the guide robot is restored to the original value, it will act in a manner of performing monitoring along the existing driving path. In the latter case, map data of a new driving path can be generated and learned. In addition, even if the state data of the guide robot is restored to the original value, the driving path for monitoring can continue to be generated based on the state data acquired in real time.

[0116] Specifically, the processor 180 of the guide robot 100 may change / set the reference POI used for monitoring to be different according to the received or acquired state information of the guide robot.

[0117] Alternatively, the processor 180 of the guide robot 100 may determine the monitored object and set a driving path for tracking the object based on the received or acquired state information of the guide robot.

[0118] Alternatively, if the processor 180 of the guide robot 100 determines that the remaining battery power is insufficient to monitor the entire monitoring area based on the received or acquired status information of the guide robot, the processor 180 will be set to travel only in selected security-required locations or narrower areas based on specified benchmarks.

[0119] Next, an example of changing the travel path for monitoring based on specific state data of the guiding robot will be described.

[0120] Figure 5 It is a diagram showing that a monitoring function is performed in consideration of the field angle range of a camera in a guide robot related to the present invention.

[0121] If the monitoring function of the guide robot 100 is executed, the camera 121 will be activated and operated in the monitoring mode to monitor the surroundings. In the monitoring mode, the guide robot 100 needs to be able to confirm the entire monitoring area according to the field of view of the camera monitoring the surroundings while driving in the designated monitoring area.

[0122] Thus, the processor 180 of the guiding robot 100 can determine a driving path for monitoring based on the field of view angle range of the activated camera 121 to monitor the entire monitoring area.

[0123] Reference Figure 5 , considering the field of view of the camera of the guide robot 100, the driving path 512 can be set to be a spiral from the boundary of the monitoring area 510 toward the inside and the monitoring mode can be executed to cover the entire interior of the designated monitoring area 510. That is, the plurality of POIs (1, 2, 3, 4) set by the administrator are connected and regarded as the boundary of the monitoring area, and the robot travels in a manner covering the entire internal space thereof.

[0124] Afterwards, if the guide robot 100 is located at the center of the monitoring area 510, a spiral driving path is drawn from the inside of the monitoring area 510 toward the boundary in the direction opposite to the above direction and the monitoring mode is executed. This process can be repeatedly performed until the monitoring function of the monitoring area 510 is released.

[0125] On the other hand, the spiral driving path 512 is an example considering that the field of view of the camera covers the entire monitoring area 510, so driving paths of other shapes are also feasible, for example, a zigzag driving path.

[0126] As yet another embodiment, Figure 6a , Figure 6b as well as Figure 6c An example is shown in which the travel route of the monitoring area is set differently based on different states of the guide robots, specifically, a case where the battery power is low and a case where there is another guide robot that can cooperate.

[0127] The guide robot 100 of the present invention may set a travel route in the monitoring area in consideration of the remaining travel time calculated based on the remaining battery power.

[0128] When the guide robot 100 is performing the monitoring mode operation or the monitoring mode operation, if it is determined that the battery power of the guide robot 100 is insufficient, the monitoring function is performed in the power saving mode. This is because the size of the guide robot 100 has the greatest influence on the battery power consumption required for traveling.

[0129] However, even if the remaining battery power of the guide robot 100 is below the baseline value, the remaining battery power and the remaining operating time will be compared. If the result is that the estimated time required to monitor the surveillance area is very short (i.e., it is predicted that the monitoring will end immediately), the mode will not be switched to the power saving mode and the monitoring function will not be continued.

[0130] When performing the monitoring function in the power saving mode, it is advantageous to place the guide robot 100 in a position that minimizes travel and improves security, for example, a position with the highest visibility. Alternatively, the guide robot 100 may be placed at a position with the highest visibility without moving and continue to perform the monitoring function.

[0131] According to an embodiment, the processor 180 of the guide robot 100 may determine a location where more tourists gather or a specific location where more mobile populations are present (e.g., around a door, around a toilet, around an activity, etc.) as a location with higher visibility, and control the guide robot 100 to move to the location. Alternatively, according to an embodiment, the guide robot 100 may also move to a location or direction where decibels (dB) above a reference value are generated, and the processor 180 performs a monitoring function.

[0132] The guide robot 100 can compare the remaining battery power and the estimated time required to monitor the entire monitoring area. If it is determined that the remaining battery power is insufficient, it will consider the remaining driving time and perform the monitoring mode action in the power saving mode (pause driving).

[0133] To this end, the processor 180 of the guide robot 100 can execute a power saving mode according to the remaining battery power of the guide robot. As the power saving mode is executed, the travel unit 130 can be controlled to move the guide robot 100 to a specific position within the monitoring area and monitor the monitoring area.

[0134] Thereafter, guide robot 100 may stop traveling and monitor the surveillance area at the moved position or along a shorter travel path generated based on the moved position.

[0135] At this time, the specific location can be changed according to the status information collected from the monitoring area. Here, the status information may include status data (e.g., visitor density, sound, etc.) and / or environmental data (e.g., cooling and heating temperature, humidity, ground conditions, etc.) collected in the monitoring area by components or sensors installed on the guide robot.

[0136] Figure 6a As the status information, it is shown that the specific location is determined based on the density of tourists (or users) in the monitoring area.

[0137] like Figure 6a As shown, although multiple POIs (1, 2, 3, 4) of the monitoring area 510 have been set, when the power saving mode is executed due to low battery condition, a location or area G with a high density of tourists in the monitoring area 510 can be detected by a camera or a linked server, and a shorter driving path 611 can be set based on the location or area and the monitoring function can be performed, or the monitoring function can be performed while standing at any location within the location or area G (for example, POI No. 4).

[0138] Figure 6b As the status information, it is shown that the specific location is identified by detecting sound or noise having a decibel (dB) level greater than a reference value in the monitoring area.

[0139] like Figure 6bAs shown, although a plurality of POIs (1, 2, 3, 4) of the monitoring area 510 have been set, in the case where the power saving mode is executed due to a low battery condition, the sound receiving unit 122 of the guiding robot 100, for example, a microphone, can be used to detect a position or area S where the sound / noise collected in the monitoring area 510 is above a reference value in decibels (dB), and a shorter driving path 612 can be set based on the position or area and the monitoring function can be performed, or the monitoring function can be performed while standing at any position within the position or area S (for example, POI No. 2).

[0140] On the other hand, according to the embodiment, there is a case where a plurality of guide robots monitor a monitoring area. In the case where a plurality of guide robots divide a monitoring area for monitoring, it is effective that the driving path for monitoring is also adaptively changed.

[0141] Thus, the guide robot 100 of the present invention can change the driving path of the monitoring area according to the existence of other guide robots that can cooperate and / or receiving a request for cooperation.

[0142] Specifically, the processor 180 of the guide robot 100 can determine the driving path of the monitoring area to be different according to the number of collaborative robots. To this end, the guide robot 100 can communicate with other collaborative robots and receive collaborative schedule information of the same monitoring area set by the administrator from the administrator terminal or the management server.

[0143] Here, the collaborative schedule information of the monitoring area may include information related to the number of collaborative robots and the collaborative work time.

[0144] For example, the greater the number of collaborative robots, the more the monitoring area is subdivided, so that the travel path that each guiding robot needs to travel for monitoring becomes shorter.

[0145] In addition, for example, the cooperation schedule can be set differently according to time periods. For example, at 1 pm, one guide robot can be arranged to monitor, at 5 pm, the previous guide robot can be arranged to return and another guide robot can be arranged to monitor, at 2 pm, two guide robots can be arranged to monitor the monitoring area in cooperation, at 4 pm, one of the previous two guide robots can be replaced with another guide robot, and at 3 pm, an additional guide robot can be added, so that three guide robots can monitor the monitoring area.

[0146] Reference Figure 6cWhen comparing the situation in which one guide robot 100a monitors the monitoring area 510, the situation in which two guide robots 100a and 100b monitor the monitoring area 510, and the situation in which three guide robots 100a, 100b, and 100c monitor the monitoring area 510, it can be predicted that the smaller the size of the subdivided monitoring area, the shorter the driving path used for monitoring will be set.

[0147] On the other hand, in another embodiment, in the power saving mode, a request can be made to other guide robots or administrators in the surrounding area to dispatch a collaborative guide robot. When performing the monitoring function at the specific location, other guide robots can respond according to the request and approach the monitoring area, and the guide robot operating in the power saving mode can return to the charging station.

[0148] As described above, the guide robot of the present invention can take into account the field of view angle range of the surveillance camera, and can actively set the driving path of the surveillance area based on various status information such as the remaining battery power, the estimated time required to monitor the entire surveillance area, and the presence of other collaborative guide robots.

[0149] the following, Figure 7 , Figure 8a , Figure 8b , Fig. 9 as well as Fig.10 An example is shown in which a guide robot according to the present invention recognizes the occurrence of a monitoring event and executes a corresponding action while traveling in a monitoring area.

[0150] As described above, when monitoring is performed at a fixed position or along a fixed driving path, the monitoring effect is not high. In this regard, the above description describes an example of setting a more efficient driving path within a specified monitoring area. Next, an example of identifying a problem occurrence event within the monitoring area and performing tracking to handle the problem occurrence event is described.

[0151] Here, the problem occurrence event is detected in the monitoring area, but even if leaving the monitoring area, tracking can be performed. As the tracking is performed and the event is processed, the guiding robot 100 of the present invention will return to the monitoring area again and continue to perform the existing monitoring function.

[0152] By detecting and tracking problem occurrence events as described above, the guidance robot 100 can more effectively monitor predicted problem conditions and handle them appropriately, thereby preventing possible problems in advance.

[0153] Specifically, according to an embodiment, while guiding the robot 100 to travel in the monitoring area, the processor 180 can identify the occurrence of a pre-set monitoring event (or a problem occurrence event), and can execute an action corresponding to the identified monitoring event, and then perform monitoring of the monitoring area.

[0154] Here, the pre-set monitoring event may refer to a situation scenario related to at least one of visitor status monitoring, visitor epidemic prevention monitoring, protected area monitoring, and status monitoring within a building.

[0155] For example, visitor status monitoring may include lost children monitoring, visitor monitoring who needs guidance, and pet monitoring.

[0156] Specifically, when the guide robot 100 finds a child crying alone through the camera, the guide robot 100 will regard it as a situation that requires lost child monitoring, and the guide robot 100 will autonomously approach the child and display a screen for finding parents through the display 151. In addition, if a tourist who repeatedly looks around is found in the monitoring area, it will be identified as a situation that requires tourist monitoring that requires guidance, and the guide robot 100 will approach the tourist and speak a pre-set guidance voice (for example, "What help do you need?"). In addition, when a pet is found without a leash or a pet that may approach a no pet zone through the camera, it will be identified as a pet monitoring situation, and voice guidance will be performed or a corresponding action will be performed by posing a posture to prevent approach.

[0157] In addition, for example, epidemic prevention monitoring of tourists may include temperature measurement of tourists, monitoring of compliance with epidemic prevention standards, etc.

[0158] Specifically, when an area where tourists gather is sensed, the guide robot 100 can monitor the faces of tourists and measure their body temperatures through, for example, thermal imaging cameras, or can detect tourists who are not wearing necessary epidemic prevention masks through cameras, track the monitored tourists, and guide them to perform feedback actions (for example, 'wear a mask' or 'notify the administrator', etc.) through the display 151.

[0159] In addition, for example, protection area monitoring may include monitoring of approach / over-approach to a protected object, monitoring of approach to a prohibited entry area / enclosed area, etc.

[0160] Specifically, when a child approaches a work of art in an art museum or runs, the guide robot 100 will regard it as a situation that requires monitoring of approaching / over-approaching a protected object, and will approach the protected object and monitor whether the child is over-approaching through the camera, and will issue a warning or perform a corresponding action by posing a blocking posture when approaching. In addition, if the guide robot 100 identifies a situation that requires monitoring of approaching a prohibited entry area / enclosed area based on the map data of the monitored area and the movement direction of the visitor monitored by the camera, it will warn of the approach through voice guidance or the like.

[0161] In addition, for example, status monitoring within a building may include slip monitoring, cleaning status monitoring, damage status monitoring, etc.

[0162] Specifically, the guiding robot 100 can identify building entrances and toilets where slippery conditions may occur on rainy days as situations requiring slippery monitoring, and warn tourists of ground conditions (e.g., moisture, etc.) through sound or guiding voice.

[0163] Reference Figures 7 to 9 , showing the situation in which epidemic prevention monitoring is performed on tourists due to an incident that occurs while the guide robot 100 is monitoring the monitoring area, as well as the corresponding tracking and corresponding actions.

[0164] like Figure 7 As shown, when the guide robot 100 repeatedly travels in sequence at the set POIs (1, 2, 3, 4) (refer to the figure mark 711) and performs monitoring mode actions on the monitoring area 710, it can identify the situation of tourist gathering and start tourist epidemic prevention monitoring.

[0165] So, if Figure 8a As shown, due to the problem, the monitoring POIs (5, 6, 7) used to perform visitor epidemic prevention monitoring (e.g., temperature measurement, whether to wear a mask, etc.) are reset. Afterwards, the guide robot 100 can calculate the coordinates P1, P2, P3 of the new POI based on the current position.

[0166] Here, the necessary information for calculating the monitoring POI may include location information of tourist gatherings, distance information based on the resolution of a camera (eg, a general camera or a thermal imaging camera), and field of view angle range information of the camera.

[0167] At this time, if Figure 8bAs shown, the guide robot 100 will not move to the first position P1 (refer to the reference numeral 811) of the newly set monitoring POI at the existing set POI (1, 2, 3, 4) and perform monitoring for tourist epidemic prevention monitoring. After performing epidemic prevention monitoring within the field of view angle 801 at position 5, it moves to position 6 P2. After that, after performing epidemic prevention monitoring within the field of view angle 802 at position 6, it moves to position 7 P3.

[0168] That is, the processor 180 of the guide robot 100 can calculate the POI position for performing the action corresponding to the identified monitoring event based on the occurrence location of the situation scenario, and can set the next driving path based on the calculated POI position and the current position of the guide robot.

[0169] If it is confirmed that there is no abnormality in the monitoring results of the monitoring event or the problem occurrence event, it will move to one of the existing POIs (1, 2, 3, 4) based on the current position and execute the monitoring mode.

[0170] When the monitoring result of the monitoring event or the problem occurrence event is confirmed to be abnormal, the corresponding action of the guidance robot 100 is as follows.

[0171] like Fig. 9 (a) and Fig. 9 As shown in (c), the guide robot 100 can sequentially visit the POI positions (5, 6, 7) (refer to the reference numeral 911) that are reset for epidemic prevention monitoring of tourist G, and perform body temperature measurement monitoring within the field of view angle range 901, 902 at each position. Fig. 9 As shown in (c), at least one monitoring object OB1, OB2, OB3 is confirmed, and OB3 with a higher score (for example, not wearing a mask or having a higher body temperature measurement result) is selected as the tracking object 931.

[0172] The guide robot 100 may display the confirmed monitoring objects OB1, OB2, OB3 and information 910 related to the tracking object 931 through the display 151. At this time, the displayed information 910 may include a feedback action of the tracking object 931 and a response request.

[0173] According to an embodiment, the guiding robot 100 can approach the tracking object 931 and track (follow) the movement of the tracking object 931 while outputting a guiding voice to perform a feedback action (for example, wearing a mask, notifying an administrator, leaving a building, etc.), or issue a warning voice to notify people around the moving path of the tracking object 931.

[0174] At this time, even if leaving the monitoring area, the guide robot 100 can continue to track the tracking object 931.

[0175] If it is confirmed that the tracking object 931 performs the feedback action, the guide robot 100 returns to the monitoring area and monitors the monitoring area again with a driving path changed based on the existing driving path or the status information of the robot.

[0176] On the other hand, if it is confirmed that the tracking object 931 has not performed a feedback action after a predetermined time has passed, the guidance robot 100 transmits this fact to a terminal such as an administrator and makes a call.

[0177] Fig.10 It shows the corresponding tracking and corresponding actions when the monitoring of approach / over-approach to the protected object is performed due to a problem occurrence event during the monitoring of the guide robot 100 in the monitoring area.

[0178] The monitoring area 1010 may be set as a specified space in a pavilion or art gallery with expensive works or exhibits. In this case, if a situation in which a child or the like approaches an expensive work or exhibit is identified, tracking and corresponding actions for monitoring approach / over-approach to the protected object are performed to prevent theft or damage.

[0179] like Fig.10 As shown, when the guide robot 100 performs monitoring mode actions while repeatedly traveling at designated POIs (1, 2, 3, 4) in the monitoring area 1010, if a situation in which a visitor OB (e.g., a child) approaches a protected object T (e.g., a work) is identified by a camera, etc., the robot moves to a position close to the protected object T and tracks the visitor OB.

[0180] To this end, although not shown in the figure, the guide robot 100 may calculate a monitoring POI capable of measuring the distance between the protection target T and the tourist OB being tracked, and may move at the calculated monitoring POI.

[0181] Afterwards, the guide robot 100 can output a warning notification of approach to the tracking target tourist OB through the display 151 and / or the sound output unit. In addition, the guide robot 100 can record and store the image corresponding to the tracking through the camera during the tracking. In addition, in the case where the tracking target tourist OB ignores the warning notification and further approaches the protection object T, the guide robot 100 can gradually strengthen the warning notification in inverse proportion to the distance between the protection object T and the tracking target tourist OB. If the tourist OB damages the protection object T and disappears, a stronger warning sound will be output and the notification and recorded image will be immediately transmitted to the terminal of the administrator, etc.

[0182] As described above, the guiding robot of the present invention is capable of performing more segmented or specialized monitoring, such as epidemic prevention, monitoring of protected areas, etc. When problems or events occur, tracking and appropriate corresponding actions can be performed on related objects.

[0183] Fig.11 1 is another flowchart for explaining the method of guiding the robot related to the present invention. Fig.11 The steps and Figure 4 The same is performed by the processor 180 of the guiding robot 100. In addition, Fig.11 The process may be repeatedly performed during the period of guiding the robot 100 to perform the monitoring function, and the guiding robot 100 may Fig.11 process and performs AI learning, updating and / or building a database based on the monitoring results of the monitored area.

[0184] Reference Fig.11 , first, the guide robot 100 may activate a camera according to an input or autonomous judgment, thereby executing a monitoring mode (S110).

[0185] Here, the camera may include a depth camera, a two-dimensional camera, a three-dimensional camera, a thermal imaging camera, etc., in addition to a general camera, for special monitoring. In addition, as the monitoring mode of the guiding robot is executed, the camera may be activated to perform a permanent recording function.

[0186] If the monitoring mode is executed, the guide robot 100 receives or acquires the state information of the travel space while traveling in the monitoring area ( S120 ).

[0187] Here, the state information of the driving space may refer to state data and environment data related to abnormality detection of the driving space.

[0188] In addition, the state data may include data related to the ground state (e.g., potholes, slippage, inclination) of the driving space, data related to the state of the structure (e.g., damage to the structure, etc.). The environmental data may include data related to the cooling and heating temperature, humidity, congestion, noise level, etc. of the driving space.

[0189] The state information of such driving space can be received in various ways and manners, such as through a state information-related sensor 143 or a tilt sensing sensor 144 (e.g., an inertial measurement unit (IMU) sensor) of the detection unit 140 of the guiding robot, environmental sensors such as a temperature sensor and a humidity sensor, a communication unit 110, a camera 121 and / or a sound receiving unit 122 for acquiring surrounding sounds.

[0190] Next, the guide robot 100 may determine a next driving direction based on the received state information of the driving space ( S130 ).

[0191] According to an embodiment, the next traveling direction of the guide robot may be determined as a direction related to a position where an abnormality is detected as a result of analysis of the received state data of the traveling space.

[0192] Then, as described above, the guide robot 100 may confirm a monitoring object corresponding to the state information of the travel space collected while traveling in the determined travel direction ( S140 ), and may perform a corresponding action related to the confirmed monitoring object ( S150 ).

[0193] Specifically, while guiding the robot 100 to travel and monitor in the monitoring area, it can confirm the cooling and heating temperature of the travel space and detect cooling and heating anomalies, and can track the next path based on the location of the detected anomaly, thereby detecting the location that causes heat loss.

[0194] Alternatively, when guiding the robot 100 to travel and monitor in the monitoring area, it can detect abnormalities such as potholes, slippage, and tilt on the ground and establish a database, or it can track multiple other locations based on the locations where the abnormalities are detected and establish a database, and then transmit notifications to terminals such as administrators.

[0195] Fig.12 , Fig.13 , Fig.14 The diagram is used to explain a method of detecting abnormality in the cooling and heating states and confirming the monitored object while the guide robot 100 travels and monitors the monitored area, and corresponding operations related thereto.

[0196] Reference Fig.12 , the guide robot 100 sequentially visits the coordinates P1, P2, P3, and P4 of the set plurality of POIs (1, 2, 3, and 4) while monitoring the monitoring area 1210 (1211), and can measure the cooling and heating temperatures as the state information of the driving space. At this time, the cooling and heating temperatures can be measured using a temperature sensor provided in the guide robot 100, or can be performed by communicating with a plurality of temperature measuring devices provided around the POIs (1, 2, 3, and 4).

[0197] On the other hand, the detection of cooling and heating anomalies as described above can be initiated based on the voice of a tourist captured by a microphone (e.g., 'too hot', etc.) or based on the actions of multiple tourists observed by a camera (e.g., one or more tourists fanning themselves or taking off clothes, etc.).

[0198] The temperature value measured at a specific POI (No. 3) is compared with the temperature values ​​measured at other POIs (1, 2, 4). If the difference is above the specified range (for example, the difference between the minimum / maximum temperature values ​​is more than 2 degrees), it is detected as an abnormal state, and the coordinates P3 of the POI are determined as the location of the abnormality detection.

[0199] After that, the guide robot 100 does not move to position 1 again (revisit) to monitor the monitoring area 1210, but re-determines the next driving direction based on the position P3 where the abnormality is detected. This is to find the path and cause of the heat loss.

[0200] The guide robot 100 may calculate a search POI for searching a location where cooling and heating abnormalities are detected or a location where heat loss occurs.

[0201] In other words, the guiding robot 100 can determine the path and cause of heat loss by obtaining information related to predicted locations (e.g., door and / or window locations) where heat loss may occur based on temperature values ​​of multiple locations and building map data within the driving space (including the entire / partial space of a building outside the monitoring area).

[0202] Reference Fig.13 First, the guide robot 100 moves to the position 3 where the abnormality of cooling and heating is detected for the first time. Then, the monitoring object, that is, the candidate POI for confirming the temperature value is set at the current position 3. Fig.13 It is shown in FIG. 5 that positions 5 and 6 are set as candidate POIs.

[0203] Afterwards, the processor 180 of the guide robot 100 selects a candidate POI with a high expected weight of abnormality detection from the set candidate POIs as the search POI. Here, the expected high weight of abnormality detection may mean, for example, that a location with a higher temperature roughly detected by the thermal imaging camera of the guide robot 100 is given a higher weight score.

[0204] The candidate POI No. 6, which has a higher expected weight for abnormality detection, is selected as the search POI. After that, the robot 100 is guided to move to the position No. 5 and measure the temperature value. The measurement result is compared with other POIs (1, 2, 4) in the monitoring area 1310. If a cooling or heating abnormality is detected, the setting of the candidate POI and the selection of the next search POI (for example, No. 7) are repeated again based on the position.

[0205] According to an embodiment, at least a portion of the search POI or the next search POI is located outside the monitoring area.

[0206] According to an embodiment, the processor 180 of the guide robot 100 drives to the search POI and accumulates the state information of the driving space, and detects the next search POI based on the collection result. At this time, the guide robot 100 can connect to the search POI and generate a heat map, and can determine a direction with a higher weight for determining the next search POI based on the generated heat map.

[0207] At this time, the guiding robot 100 can identify positions 3, 5, and 7 as abnormal detection paths and further approach the cause of the heat loss.

[0208] The guide robot 100 may output information related to the corresponding action related to the confirmed monitoring object (e.g., temperature value, heat loss path) to the display 151 of the guide robot. On the other hand, the guide robot 100 may process autonomously (e.g., transmit a signal to close the window), or, if it is a case where it can be remotely controlled, perform the corresponding action. However, if it is a case where it cannot be processed autonomously, it requests help from the surrounding area or the administrator.

[0209] Reference Fig.14 After confirming that 'the window is open W' as the cause of heat loss, the guiding robot 100 can output a guiding voice to guide the surrounding to close the window, and / or can output a guiding message 1410 (for example, 'Please close the window, heat loss is occurring') guiding the corresponding action through the display 151.

[0210] According to an embodiment, if it is confirmed that there is no feedback response 1411 regarding the guidance voice or displayed information (for example, closing the window) within a preset time, the guidance robot 100 will transmit the confirmation result of the monitored object and the corresponding action related to the confirmed monitored object to a preset administrator terminal 1412 through the communication unit.

[0211] As a result, specific areas where heat loss occurs during cooling and heating in a building can be detected more quickly than a building management system (BMS), and the cause of the heat loss can be quickly resolved by requesting help from surrounding visitors.

[0212] As yet another embodiment, Fig.15a , Fig.15b as well as Fig.15c This is a diagram for explaining an example of a method of confirming the management status of a building when an abnormal tilt of the floor of a building is detected during monitoring of a surveillance area.

[0213] First, refer to Fig.15aWhen guiding the robot 100 to repeatedly travel at the set POI (1, 2, 3, 4) and monitor the monitoring area 1510, the ground inclination state at each POI (1, 2, 3, 4) can be measured by the inclination sensing sensor 144 of the detection unit 140.

[0214] At this time, the inclination sensing sensor 144 may include, for example, an IMU sensor and a depth camera. Here, the IMU sensor is a sensor that can use an accelerometer, a gyroscope, and a magnetometer to measure the current position and posture of the guide robot 100. Generally, three gyroscopes and three accelerometers are configured and six degrees of freedom (DOF) (X, Y, Z, roll, pitch, yaw) values ​​are provided as measurement values.

[0215] Specifically, if Fig.15b As shown, the guiding robot 100 measures the values ​​of the inclination of the guiding robot 100 itself relative to the three axes (first measurement values) through the IMU sensor, thereby performing localization. In addition, the guiding robot 100 analyzes the image plane and the ground plane image acquired by the depth camera and calculates the inclination of the ground (second measurement value). The second measurement value is also used as a correction value for distinguishing obstacles from the ground (floor).

[0216] The guiding robot 100 receives detection values ​​through the IMU sensor and the depth camera at intervals of about 100 msec and obtains the first measurement value and the second measurement value.

[0217] The first measurement value obtained represents the tilt degree of the guide robot 100 at the current position, and the second measurement value represents the tilt degree of the floor in front of the depth camera.

[0218] The guide robot 100 may confirm the inclination of the ground at the current position based on the first measurement value and the second measurement value, and the inclination may be cumulatively stored in the memory 170 or an external server and a database may be established.

[0219] Refer again Fig.15a For example, when a specific POI (No. 2) detects a ground slope exceeding a reference value, tracking is performed to detect the ground slope of the entire / part of the building outside the monitoring area 1510.

[0220] Therefore, if Fig.15cAs shown, a plurality of additional POIs (5, 6, 7, 8, 9) for abnormality detection are set outside the monitoring area 1510, and the inclination of the ground is cumulatively detected by the IMU sensor and the depth camera while driving through the additional POIs (5, 6, 7, 8, 9) in sequence. Figure 12 to Figure 14 Different from the cooling and heating abnormality detection described above, the selection of the additional POI is not achieved by repeatedly selecting candidate POIs -> selecting detection POIs while generating a detection path, but by selecting multiple locations separated from each other as additional POIs (5, 6, 7, 8, 9) to detect the overall inclination of the entire / part of the building.

[0221] On the other hand, unless it is an emergency, the detection of the ground inclination can be postponed. For example, the monitoring area can be monitored during the daytime, and then the additional POIs (5, 6, 7, 8, 9) can be driven and measured during the nighttime to detect the ground inclination.

[0222] As described above, according to some embodiments of the present invention, the guide robot and its operation method can take into account the field of view of the monitoring camera, and can actively set the driving path of the monitoring area or change the existing driving path based on various status information such as the remaining battery power, the estimated time required to monitor the entire monitoring area, and the existence of other guiding robots that can cooperate. In this way, more effective and intelligent monitoring can be performed instead of uniform monitoring. In addition, more subdivided or specialized monitoring of domain monitoring such as epidemic prevention and protection areas can be achieved, and when a problem occurs, tracking and appropriate corresponding actions can be performed on related objects. In addition, as an appropriate corresponding action, a feedback response can be requested, and an administrator can be notified if there is no response to the request. In this way, more appropriate detailed abnormality detection and corresponding actions can be achieved according to the situation.

[0223] The additional scope to which the present invention can be applied will become clearer through the following detailed description. However, those skilled in the art can clearly understand the various changes and modifications within the thought and scope of the present invention, and therefore it should be understood that the detailed description and the specific implementation of the preferred embodiment of the present invention are only exemplary.

[0224] The features, structures, effects, etc. described in the above embodiments are included in at least one embodiment of the present invention, and are not necessarily limited to only one embodiment. Further, the features, structures, effects, etc. exemplified in each embodiment can be combined or deformed and implemented in other embodiments by a person of ordinary skill in the art to which the embodiments belong. Therefore, the contents related to such combinations and deformations should be interpreted as being included in the scope of the present invention.

[0225] In addition, the above description is centered on the embodiment, but this is only an example and does not limit the present invention. For those skilled in the art to which the present invention belongs, various modifications and applications not exemplified above can be made within the scope of the essential characteristics of the present embodiment. For example, the embodiments specifically show that individual constituent elements can be modified and implemented. Moreover, the differences related to such modifications and applications should be interpreted as being included within the scope of the present invention as defined in the appended claims.

Claims

1. A guiding robot, wherein: include: A camera, when activated, photographs the periphery of the guide robot; A traveling unit, which moves the guide robot; A detection unit, which collects status information of the guiding robot; as well as a processor, electrically connected to the camera, the traveling unit, and the detecting unit; As the monitoring mode is executed, the processor activates the camera. In the monitoring mode, the processor receives status information of the guide robot while the guide robot is traveling in the monitoring area, and determines the driving path of the monitoring area based on the received status information of the guide robot.

2. The guiding robot according to claim 1, wherein: The processor determines the driving path based on the field of view angle range of the camera so as to monitor the entire monitoring area.

3. The guiding robot according to claim 1, wherein: The state information of the guide robot includes at least one of a remaining battery charge of the guide robot, an estimated time required to monitor the entire monitoring area, and the presence of other cooperative guide robots.

4. The guiding robot according to claim 3, wherein: The processor executes a power saving mode according to the remaining battery power of the guide robot, and as the power saving mode is executed, the processor controls the travel unit to move the guide robot to a specific position within the monitoring area and monitor the monitoring area; The specific location changes according to the status information collected from the monitoring area.

5. The guiding robot according to claim 1, wherein: During driving in the monitoring area, the processor identifies the occurrence of a preset monitoring event, executes an action corresponding to the identified monitoring event, and then monitors the monitoring area.

6. The guiding robot according to claim 5, wherein: The pre-set monitoring event is a situation scenario related to at least one of visitor status monitoring, visitor epidemic prevention monitoring, protected area monitoring, and status monitoring within a building.

7. The guiding robot according to claim 6, wherein: The processor calculates a point of interest position for executing an action corresponding to the identified monitoring event based on the occurrence location of the situation scenario, and sets a subsequent driving path based on the calculated point of interest position and the current position of the guide robot.

8. The guiding robot according to claim 6, wherein: Also included is a touch screen that displays information related to the identified monitoring event; The processor is electrically connected to the touch screen and requests a feedback response corresponding to an action corresponding to the identified monitoring event through the touch screen.

9. The guiding robot according to claim 1, wherein: The detection unit collects state information of the driving space; In the monitoring mode, the processor determines the next driving direction based on the state information of the driving space collected during driving in the monitoring area, confirms the monitoring object corresponding to the state information of the driving space collected while driving in the determined driving direction, and performs corresponding actions related to the confirmed monitoring object.

10. The guiding robot according to claim 9, wherein: The state information of the driving space is state data including at least one of temperature data, slip data, and inclination data of the driving space.

11. The guiding robot according to claim 10, wherein: The next travel direction of the guide robot is determined as a direction associated with a position where an abnormality is detected, and the position where an abnormality is detected in an analysis result of the state data of the travel space.

12. The guiding robot according to claim 11, wherein: The processor sets candidate interest points for confirming the monitored object based on the position of the anomaly detection, and selects an interest point with a higher expected weight of anomaly detection among the set candidate interest points as a search interest point; The vehicle drives toward the searched interest point and accumulates and collects state information of the driving space, and detects the next searched interest point based on the collected result.

13. The guiding robot according to claim 12, wherein: At least a portion of the search point of interest or the next search point of interest is located outside the monitoring area.

14. The guiding robot according to claim 9, wherein: The device also includes a touch screen for displaying information related to the confirmed corresponding action related to the monitored object.

15. The guiding robot according to claim 14, wherein: The guiding robot further includes a communication unit; After the processor confirms that there is no feedback response regarding the displayed information within a preset time, the processor transmits the confirmation result of the monitored object and the corresponding action related to the confirmed monitored object to a preset administrator terminal through the communication unit.

16. A method for guiding the movement of a robot, wherein: include: Steps to activate the camera of the guide robot and execute surveillance mode; In the monitoring mode, the step of acquiring status information of the guide robot while traveling in the monitoring area; as well as Based on the acquired state information of the guiding robot, the driving path of the monitoring area is determined into different steps.

Images