Robot and sound source detection method by robot

Abstract

The present invention relates to a robot and a sound source detection method of a robot. A robot according to an embodiment of the present invention comprises: a driving unit that moves a main body of the robot; a microphone that receives an external sound; and a control unit that determines a direction in which an external sound is detected for each position of the robot while moving the main body of the robot, extracts an intersection of external sound detection directions determined from at least two positions of the robot, and estimates a sound source location of the external sound using at least one of the extracted intersections.

Description

Robots and methods for detecting the source of sound from robots

[0001] The present invention relates to a robot and a method for detecting the source of sound of the robot.

[0002] Robots were developed for industrial use and have played a role in factory automation. Recently, the scope of robotics applications has expanded, with developments in medical and aerospace robots, as well as domestic robots for general use. Among these robots, those capable of autonomous navigation are called mobile robots. A representative example of a mobile robot used in the home is a robot vacuum cleaner.

[0003] Several technologies are known for detecting the environment and user surrounding a robot vacuum cleaner through various sensors installed in the robot vacuum cleaner. Furthermore, technologies are known for allowing a robot vacuum cleaner to learn and map its driving area and determine its current location on the map. Robot vacuum cleaners are known for cleaning their driving area in a preset manner.

[0004] Currently, with the advancement of artificial intelligence technology, it is being applied in various fields. Following this trend, deep learning technology is also being applied in the field of acoustic recognition. Robots (or acoustic recognition devices) are emerging that detect various acoustic signals, or sound events, using various deep learning models trained with augmented data from diverse acoustic environments.

[0005] One object of the present invention is to provide a robot capable of searching for a sound source in an optimized manner and a control method thereof.

[0006] Another object of the present invention is to provide a robot capable of accurately detecting a sound source while reducing the search time, and a control method thereof.

[0007] According to one embodiment of the present invention for achieving the above or other purposes, a robot includes a driving unit configured to move a main body of the robot, a microphone configured to receive external sounds, and a control unit configured to determine a direction in which external sounds are detected for each position of the robot while moving the main body of the robot, extract an intersection of external sound detection directions determined at the positions of at least two robots, and estimate a location of a sound source of the external sounds using at least one extracted intersection.

[0008] In an embodiment, the control unit is characterized in that it extracts the at least one intersection point using an external sound detection direction determined at a plurality of robot locations, and determines a representative value for the at least one intersection point using a preset filter.

[0009] In an embodiment, the preset filter is characterized by including at least one of a Kalman filter, a particle filter, and DBSCAN (Density-based spatial clustering of application with noise).

[0010] In an embodiment, the control unit is characterized in that it determines a cluster of a plurality of intersection points of external sound detection directions determined at a plurality of robot locations, and determines a sound source location of the external sound based on probability based on the determined cluster.

[0011] In an embodiment, the control unit is characterized in that, when the location of the sound source of the external sound is estimated, the control unit controls the driving unit to perform a search operation to determine the location of the sound source of the external sound based on the estimated location.

[0012] In an embodiment, the control unit is characterized in that it sets a search point to drive in a predetermined area adjacent to the sound source location centered on the sound source location of the estimated external sound.

[0013] In an embodiment, the control unit is characterized in that it controls the driving unit so that the robot drives while passing through the search point, and searches for the location of the sound source of the external sound while driving while passing through the search point.

[0014] In an embodiment, the control unit is characterized in that it determines the sound source location of the external sound under different conditions in a first mode for estimating the sound source location of the external sound and a second mode for searching the sound source location of the external sound while passing through the search point.

[0015] In an embodiment, the control unit is characterized in that the number of clusters of a plurality of intersections of external sound detection directions is greater in the second mode than in the first mode, and a closer cluster distribution is formed.

[0016] In an embodiment, the control unit is characterized in that, when driving to the search point is completed, the location of the sound source of the external sound is determined.

[0017] In an embodiment, the system further includes a memory, and the control unit is characterized in that it stores information including at least one of the estimated sound source location of the external sound, the time at which the external sound was received, the type of the external sound, and the function performed by the robot in the memory.

[0018] In an embodiment, the control unit is characterized in that, when an external sound is received, it recognizes the type of the external sound, and, if the type of the external sound corresponds to a preset type, it determines whether corresponding information exists among the information stored in the memory.

[0019] In an embodiment, the control unit is characterized in that, if the corresponding information exists in the memory, the control unit moves the robot to a location estimated as the sound source location of the external sound using the corresponding information, and enters a mode for searching for the sound source location of the external sound while passing through a search point based on the estimated location.

[0020] In an embodiment, the control unit is characterized in that, when the location of the sound source of the external sound is detected, information including at least one of the location of the sound source of the external sound, the time at which the external sound was received, the type of the external sound, and the function performed by the robot is updated in the memory.

[0021] According to at least one of the embodiments of the present invention, the present invention can provide a method in which a robot can estimate a sound source while moving, and perform another search drive by driving around the estimated sound source for accurate judgment, thereby improving the accuracy of the sound source location.

[0022] Figure 1 is a block diagram illustrating the configuration of a robot according to an embodiment of the present invention.

[0023] Figure 2 is a flowchart for explaining a representative sound source search method of the present invention.

[0024] Figures 3, 4 and 5 are conceptual diagrams for explaining the sound source search method examined in Figure 2.

[0025] Figure 6 is a flowchart illustrating a method for searching for a sound source according to another embodiment of the present invention.

[0026] Figure 7 is a conceptual diagram explaining the sound source search method examined in Figure 6.

[0027] FIG. 8, FIG. 9, FIG. 10, FIG. 11, FIG. 12, FIG. 13, FIG. 14, FIG. 15, FIG. 16, FIG. 17 and FIG. 18 are conceptual diagrams for explaining a method in which a robot performs a corresponding function based on the location, time and type of sound after searching for the source of sound according to the present invention.

[0028] It should be noted that the technical terms used herein are used merely to describe specific embodiments and are not intended to limit the present invention. Furthermore, singular expressions used herein include plural expressions unless the context clearly dictates otherwise. The suffixes "module" and "part" used in the following description for components are assigned or used interchangeably solely for the convenience of writing the specification, and do not in themselves have distinct meanings or roles.

[0029] In this specification, the terms “comprises” or “includes” should not be construed to necessarily include all of the components or steps described in the specification, and some of the components or steps may not be included, or additional components or steps may be included.

[0030] In addition, when describing the technology disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the technology disclosed in this specification, the detailed description is omitted.

[0031] In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention. In addition, not only each embodiment described below, but also a combination of embodiments may correspond to the spirit and technical scope of the present invention as modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention.

[0032] Figure 1 is a block diagram illustrating the configuration of a robot according to an embodiment of the present invention.

[0033] A robot (or, acoustic recognition device) (10) according to an embodiment of the present invention may be configured to include a control unit (180), a communication unit (110), an input unit (120), a driving unit (130), a sensing unit (140), an output unit (150), a memory (170), and an interface unit (160) connected to the control unit (180). The components illustrated in FIG. 1 are not essential for implementing the robot (10), and thus, the robot (10) described in this specification may have more or fewer components than the components listed above.

[0034] More specifically, among the above components, the communication unit (110) may include one or more modules that enable wireless communication between the robot (10) and a wireless communication system, between the robot (10) and at least one peripheral device, or between the robot (10) and an external server.

[0035] This communication unit (110) may include at least one of a wireless Internet module (111), a short-range communication module (112), and a location information module (113).

[0036] The wireless Internet module (111) refers to a module for wireless Internet access, and can be built into or externally mounted on the robot (10). The wireless Internet module (111) is configured to transmit and receive wireless signals in a communication network according to wireless Internet technologies.

[0037] The short-range communication module (112) is for short-range communication and can support short-range communication using at least one of Bluetooth, RFID, infrared communication, UWB, ZigBee, NFC, Wi-Fi, Wi-Fi Direct, and Wireless USB technologies. This short-range communication module (112) can support wireless communication between the robot (10) and a wireless communication system, between the robot (10) and a peripheral device, or between the robot (10) and a network where an external server is located through a short-range wireless communication network (Wireless Area Network).

[0038] The location information module (113) is a module for obtaining the location (or current location) of the robot (10), and representative examples thereof include a GPS (Global Positioning System) module or a WiFi (Wireless Fidelity) module. For example, if the robot (10) utilizes a GPS module, it can obtain the location of the robot (10) using signals transmitted from GPS satellites. As another example, if the robot (10) utilizes a Wi-Fi module, it can obtain the location of the robot (10) based on information from a wireless AP (Wireless Access Point) that transmits or receives wireless signals with the Wi-Fi module.

[0039] As needed, the location information module (113) may perform the function of any of the other modules of the communication unit (110) to obtain data regarding the location of the robot (10) as a substitute or additionally. The location information module (113) is a module used to obtain the location (or current location) of the robot (10), and is not limited to a module that directly calculates or obtains the location of the robot (10).

[0040] The input unit (120) may include a camera (121) or a video input unit for inputting a video signal, a microphone (122, hereinafter referred to as a microphone) or an audio input unit for inputting an audio signal, and a user input unit (123, for example, a touch key, a mechanical key, etc.) for receiving information from a user. Voice data or image data collected by the input unit (120) may be analyzed and processed into a user's control command.

[0041] Meanwhile, the camera (121) and microphone (122) can each collect video and audio signals from around the robot (10). In this case, the camera (121) may be equipped with one or more image sensors and can process image frames, such as still images or moving images, obtained by the image sensors. The processed image frames can then be stored in the memory (170).

[0042] In addition, the microphone (122) can sample external acoustic signals at preset time intervals and process them into acoustic data. The processed acoustic data can be converted into data for acoustic recognition in the robot (10), for example, a frame-by-frame mel spectrogram, and the converted mel spectrogram can be used as an input for a deep learning model for acoustic recognition. Meanwhile, the microphone (122) can be implemented with various noise removal algorithms to remove noise generated in the process of receiving an external acoustic signal. The microphone (122) can be designed with a beamforming structure having directionality, and can be formed so as to further improve the reception rate of acoustic information in a specific direction. In this case, the specific direction can be the front of the robot (10).

[0043] Meanwhile, the driving unit (130) may be a component for performing the function of the robot (10) or the device equipped with the robot (10).

[0044] The above driving unit (130) may be a driving unit for performing a specific function of the robot or moving the main body of the robot.

[0045] For example, if the robot (10) is a robot vacuum cleaner that performs a cleaning function, the driving unit (130) may include at least one component for performing the cleaning function. In this case, the driving unit (130) may include a suction unit for vacuuming dust. In addition, the driving unit (130) may include at least one wheel for movement, at least one motor for controlling the rotation of the wheel, and at least one actuator. In addition, the driving unit (130) may include a brush, and at least one motor for controlling the rotation of the brush.

[0046] Meanwhile, if the robot (10) is a robot for performing a patrol function, the driving unit (130) may include a component for the patrol function. For example, the driving unit (130) may include a moving unit (e.g., a wheel, a motor, an actuator) for moving the robot (10), and may include a lighting unit including at least one light for supporting the patrol function.

[0047] Meanwhile, if the robot (10) is formed into a robot that performs a specific function, the driving unit (130) may be the robot itself. In this case, the control unit (180) may be the control unit of the robot, or the control unit of the robot may function as the control unit of the robot (10).

[0048] The above driving unit (130) is equipped with a wheel unit for driving the robot (10). The robot (10) can be moved forward, backward, left, and right or rotated by the wheel unit.

[0049] The wheel unit includes a main wheel and a sub wheel.

[0050] The main wheels are provided on each side of the robot (10) and are configured to rotate in one or the other direction according to a control signal from the control unit. Each main wheel may be configured to be driven independently of the other. For example, each main wheel may be driven by a different motor. Alternatively, each main wheel may be driven by multiple different axes provided on a single motor.

[0051] The sub-wheel supports the robot (10) together with the main wheel and is configured to assist the driving of the robot (10) by the main wheel.

[0052] The control unit controls the operation of the wheel unit, thereby enabling the robot (10) to autonomously drive on the floor.

[0053] Meanwhile, the robot (10) is equipped with a battery (not shown) that supplies power to the robot (10). The battery is configured to be rechargeable and can be configured to be detachably attached to the bottom of the robot (10).

[0054] The sensing unit (140) may include one or more sensors for sensing at least one of information within the robot (10) and information about the surrounding environment surrounding the robot (10). For example, the sensing unit (140) may include at least one of a proximity sensor (141), an illumination sensor (142), a touch sensor, an acceleration sensor, a magnetic sensor, a gravity sensor (G-sensor), a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor (IR sensor), an ultrasonic sensor, an optical sensor (e.g., a camera (see 121)), a microphone (see 122), an environmental sensor (e.g., a barometer, a hygrometer, a thermometer, a radiation detection sensor, a heat detection sensor, a gas detection sensor, etc.), a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric recognition sensor, etc.), and a sound sensor (including at least one microphone). Meanwhile, the robot (10) disclosed in this specification can utilize information sensed by at least two of these sensors in combination.

[0055] The output unit (150) is for generating output related to visual, auditory or tactile sensations, and may include a display unit (151) and an audio output unit (152).

[0056] The display unit (151) can display (output) various information (hereinafter referred to as image information) related to the operation of the robot (10) and information related to the operation of the driving unit (130). For example, the display unit (151) can display information on a recognized sound signal, an execution screen of a currently executing function, or UI (User Interface) or GUI (Graphic User Interface) information according to the displayed information.

[0057] In addition, the audio output unit (152) can output various audio data according to the control of the control unit (180). For example, the audio output unit (152) can output information about a recognized audio signal in the form of a human voice or output audio data related to a currently executing function (e.g., voice alarm, siren, etc.) according to the control of the control unit (180).

[0058] The interface unit (160) serves as a passageway for various types of external devices connected to the robot (10). The interface unit (160) may include at least one of an external charger port connected to an external charger, a wired / wireless data port, an audio output port, a video output port, and an earphone port.

[0059] Meanwhile, the normalization unit (not shown) can generate a Mel spectrum vector by collecting sound data generated by sampling an acoustic signal from a microphone (122), and can generate a Mel spectrogram by collecting the generated Mel spectrum vector. Here, Mel is a subjective perception amount of an acoustic signal according to a frequency recognizable to a human, and can be a unit expressed linearly by tuning a nonlinear frequency unit according to a frequency recognizable to a human. In other words, Mel can mean a value converted through a frequency logarithmic function (Mel scale function).

[0060] That is, a Mel spectrogram (or Mel scale spectrogram) is a spectrogram that reflects the human audible frequency range by dividing the frequency of an audio signal into a preset number of Mel units and expressing it, and can represent the spectral change of an audio signal over time.

[0061] And the normalization unit can detect the mel spectrum vector having the maximum value among the mel spectrum vectors constituting the generated mel spectrogram. And the mel spectrogram can be normalized based on the detected maximum value. For example, the normalization unit (180) can normalize the mel spectrogram by dividing the mel spectrogram by the square of the detected maximum value. And the mel spectrogram normalized by the normalization unit can be input to the artificial intelligence unit for the recognition of the acoustic signal as feature information for the recognition of the acoustic signal.

[0062] Meanwhile, the artificial intelligence department (not shown) performs the role of processing information based on artificial intelligence technology, and may include one or more modules that perform at least one of information learning, information inference, information perception, and natural language processing.

[0063] The above artificial intelligence unit can use machine learning technology to determine what kind of acoustic signal the data (hereinafter referred to as acoustic data) sampled from an acoustic signal received through a microphone (122) is. To this end, the artificial intelligence unit can infer a probability vector for each of a plurality of different acoustic signals set in advance for the acoustic data based on a deep learning model based on information learned using the machine learning technology.

[0064] Here, learning can be achieved through the aforementioned machine learning technology. This machine learning technology, based on at least one algorithm, collects and learns large amounts of information, and uses the learned information to make judgments and predictions. Information learning involves identifying the characteristics, rules, and judgment criteria of the information, quantifying the relationships between information, and using these quantified patterns to predict new data.

[0065] Meanwhile, the probability vector inferred through the deep learning model may be a probability value indicating the probability that the acoustic data is a specific acoustic signal. In other words, the probability vectors for each of the multiple different acoustic signals inferred based on the deep learning model may indicate the probabilities that the received acoustic signal may be an acoustic signal corresponding to each of the multiple different acoustic signals.

[0066] Here, the above-mentioned deep learning technology utilizes an artificial neural network (ANN) model to perform at least one of learning, judging, and processing information. The ANN may have a structure that connects layers (hidden layers) and transmits data between layers.

[0067] Meanwhile, the artificial intelligence unit may use a convolutional neural network (CNN) model to infer probability vectors of each of a plurality of different preset acoustic signals for the acoustic data. Furthermore, the convolutional neural network model (CNN) may be a model trained with augmented data from various acoustic environments, for example, at least one of which is different from the SNR (Signal to Noise) and RT 60 (Reverberation Time 60).

[0068] In addition, the artificial intelligence unit may further include an acoustic recognition decision model capable of determining a received acoustic signal as a specific acoustic signal based on probability vectors corresponding to each of a plurality of different acoustic signals inferred by the deep learning model, i.e., class-specific probability vectors (a plurality of different acoustic signals constitute a plurality of classes). Here, the acoustic recognition decision model may be a model according to the machine learning algorithm, and may be a model learned based on the class-specific probability vectors.

[0069] Meanwhile, the memory (170) can store data supporting the functions of the robot (10). The memory (170) can store a plurality of application programs (or applications) running on the robot (10), data for the operation of the robot (10), commands, data for the operation of the artificial intelligence unit (190) (e.g., hidden layers of a deep learning model or a machine learning model and weight information of each hidden layer, information for updating or learning, etc.).

[0070] In addition, the memory (170) may store data obtained by sampling the sound signal received from the microphone (122), i.e., sound data. In addition, based on the stored sound data, a Mel Spectrogram generated by the normalization unit (180), data required for normalization of the Mel Spectrogram, and information on the Mel Spectrogram normalized by the normalization unit may be stored.

[0071] Meanwhile, the control unit (180) controls each connected component and can control the overall operation of the robot (10).

[0072] For example, the control unit (180) can generate sound data by sampling an acoustic signal received through a microphone (122).

[0073] The control unit (180) can determine that an event has not occurred if the received acoustic signal is the noise signal. In this case, the control unit (180) can control the driving unit (130) not to perform a function corresponding to the occurrence of the event. Accordingly, the control unit (180) can control the driving unit (130) to continue performing the currently performing function or to maintain the current operating state.

[0074] Meanwhile, the type of the robot (10) or the robot (10) may be determined depending on the function performed by the robot (10) or the robot (10).

[0075] Meanwhile, the control unit (180) typically controls the overall operation of the robot (10). The control unit (180) can provide or process functions related to sound recognition by processing signals, data, information, etc. input or output through the components discussed above or by operating an application program stored in the memory (170).

[0076] In addition, the control unit (180) can control at least some of the components discussed in FIG. 1 to drive an application program stored in the memory (170). Furthermore, the control unit (180) can operate at least two or more of the components included in the robot (10) in combination to drive the application program.

[0077] A robot (10) according to one embodiment of the present invention may be a robot vacuum cleaner that performs cleaning. Accordingly, the robot (10) of the present invention may include a cleaning unit (not shown) for performing cleaning.

[0078] The cleaning unit is positioned in a protruding form from one side of the main body of the vacuum cleaner, and can suck in air containing dust or perform mopping. The one side can be the side on which the main body of the vacuum cleaner moves in the forward direction (F), i.e., the front of the main body of the vacuum cleaner.

[0079] This drawing shows that the cleaning unit has a shape that protrudes forward and to both the left and right sides from one side of the main body of the cleaner. Specifically, the front end of the cleaning unit is positioned at a position spaced forward from one side of the main body of the cleaner, and the left and right ends of the cleaning unit are positioned at positions spaced left and right from one side of the main body of the cleaner, respectively.

[0080] Since the main body of the vacuum cleaner is formed in a circular shape and the rear ends of the cleaning unit are formed to protrude from the main body of the vacuum cleaner to the left and right, an empty space, i.e., a gap, can be formed between the main body of the vacuum cleaner and the cleaning unit. The empty space is a space between the left and right ends of the main body of the vacuum cleaner and the left and right ends of the cleaning unit, and has a recessed shape toward the inside of the robot (10).

[0081] If an obstacle is caught in the above-mentioned empty space, the robot (10) may become stuck on the obstacle and become unable to move. To prevent this, a cover member may be arranged to cover at least a portion of the above-mentioned empty space.

[0082] The cover member may be provided on the vacuum cleaner body or the cleaning unit. In the present embodiment, the cover member is formed protrudingly on each side of the rear end of the cleaning unit, and is arranged to cover the outer surface of the vacuum cleaner body.

[0083] The cover member is positioned to fill at least a portion of the empty space, i.e., the empty space between the vacuum cleaner body and the cleaning unit. Accordingly, an obstacle can be prevented from entering the empty space, or a structure can be implemented that allows for easy removal of an obstacle even if it is trapped in the empty space.

[0084] The cover member protruding from the cleaning unit can be supported on the outer surface of the cleaner body.

[0085] If the cover member protrudes from the main body of the vacuum cleaner, the cover member may be supported by the rear of the cleaning unit. According to the above structure, when the cleaning unit collides with an obstacle and receives an impact, a portion of the impact is transmitted to the main body of the vacuum cleaner, thereby dispersing the impact.

[0086] The cleaning unit can be detachably connected to the vacuum cleaner body. When the cleaning unit is separated from the vacuum cleaner body, a mop module (not shown) can be detachably connected to the vacuum cleaner body to replace the separated cleaning unit.

[0087] Therefore, if the user wants to remove dust from the floor, he / she can attach a cleaning unit to the vacuum cleaner body, and if he / she wants to clean the floor, he / she can attach a mop module to the vacuum cleaner body.

[0088] When the cleaning unit is mounted on the main body of the vacuum cleaner, the mounting can be guided by the cover member described above. That is, by positioning the cover member to cover the outer surface of the main body of the vacuum cleaner, the relative position of the cleaning unit with respect to the main body of the vacuum cleaner can be determined.

[0089] The cleaning unit may be equipped with casters. The casters are configured to assist the movement of the robot (10) and also to support the robot (10).

[0090] Meanwhile, a robot according to one embodiment of the present invention can search for a sound source and perform cleaning for a predetermined range based on the searched sound source.

[0091] In the case of the house, the space is narrow due to various furniture and objects, so the sound is easily reflected by these objects or walls, and is acquired as noise data. Therefore, when the robot follows the direction of the angle with only the sound angle of the sound sensor, the wrong location is often detected.

[0092] Especially within hallways connecting rooms, sound reflections from the walls on either side often point toward the walls, or are sometimes captured irregularly, often resulting in failures in sound localization. To detect specific sounds within a home and extract their locations, a process for identifying and eliminating spurious noise data is crucial.

[0093] The difficulties in detecting the location of sounds in the home are as follows:

[0094] The first reason is the precision of the sound sensor.

[0095] Triangulation techniques are often used to detect the location of a sound source. By measuring the angle of the sound at two points and knowing the robot's position at those two points, the location of the sound source can be determined using the equation of two straight lines. For sound sensors, the direction of the sound is detected relative to the robot, but the distance is unknown. The angular accuracy of sound sensors is often inaccurate, typically within 5-10 degrees. Therefore, when triangulation is used, the farther the sound source is, the more inaccurate the location will be due to angular error. For the robot, however, the closer the sound source is, the more accurate the location. While increasing the number of microphones can improve this accuracy, the higher the number of microphones and the higher the CPU requirements, which leads to higher material costs.

[0096] The second reason is that most sounds in the home are acquired as noise data.

[0097] When detecting sounds such as a hair dryer, it is difficult to detect because noise such as TV or music sounds in the house acts as noise, and even when there is no TV or music sound, when detecting the sound of a hair dryer reflected from walls or objects in the house, the direction of the sound continuously changes, making it difficult to detect the exact location of the hair dryer.

[0098] The present invention can solve these problems and provide a method for finding a more accurate location of a sound source.

[0099] Fig. 2 is a flowchart for explaining a representative sound source search method of the present invention, and Figs. 3, 4, and 5 are conceptual diagrams for explaining the sound source search method examined in Fig. 2.

[0100] The present invention recognizes the direction in which sound is generated from the robot's perspective through a Sound Source Localization (SSL) algorithm based on GCC-PHAT (Generalized Cross Corelation PHAse Transform) while the robot is moving, and accumulates the directions of sounds recognized while moving to estimate the intersection point of the direction results.

[0101] The present invention can estimate the location of sound by analyzing the distribution of intersections and extract the location as x, y coordinates based on a map already recognized by the robot.

[0102] The present invention may include a mobile robot capable of determining a location for implementing such a method and at least one microphone (preferably four microphones).

[0103] The present invention can be used to configure and use a robot-driven service by applying the robot control method described in this specification to an actual robot application, identifying the location of an object through the generated sound, and recognizing changes in the user and the surrounding environment.

[0104] The present invention reduces errors by utilizing intersection points for sound source directions continuously detected by a robot while moving and estimating representative values ​​for these intersection points, and can be competitive in material costs by using only one microphone array.

[0105] In addition, the present invention uses a noise-resistant GCC-PHAT (Generalized Cross Correlation PHAse Transform)-based SSL, and can reduce the amount of computation by using the intersection points of the output directions as the basis.

[0106] In addition, the present invention can extract a representative value by Kalman filtering on intersection data and set a parameter according to distance to find all sound sources that appear without prior information on the number of sound sources to be found.

[0107] In addition, the present invention can redefine the exact location of the sound source by searching the area around the sound source when the sound source representative value appears.

[0108] The usability and customer satisfaction of mobile robots can be determined by when, where, and what they do, and the most important key to the autonomous behavior of robots is to accurately determine the location of sound.

[0109] The occurrence of sound can serve as an important event for performing a service, and knowing the exact location of the sound occurrence, not just the direction of the sound, can be utilized to provide better services.

[0110] To this end, the present invention continuously detects the direction in which a sound source is generated while the robot moves, and can extract intersection points by accumulating the moving path of the robot and the detected direction.

[0111] In addition, the present invention can estimate representative values ​​for intersections through a Kalman filter and add parameters for distances to find a sound source detected without prior information, and can record the representative value estimation location and time to use in service configuration.

[0112] Through this control method, the present invention allows the robot to recognize sounds occurring in the surroundings while moving, determine its location, and record the cycle of occurrence, thereby enabling the robot to move proactively and provide the user with the services he needs first, thereby increasing the necessity and satisfaction of mobile robots.

[0113] Referring to FIG. 2, the present invention can perform sound source localization (SSL) (S210).

[0114] As previously discussed, the robot of the present invention may include a driving unit (130) configured to move the main body of the robot, a microphone (122) that receives external sounds, and a control unit (180) that determines a direction in which external sounds are detected for each position of the robot while moving the main body of the robot, extracts an intersection of external sound detection directions determined at least at two positions of the robot, and estimates a location of a sound source of the external sounds using at least one of the extracted intersections.

[0115] The above microphone (122) may include at least one microphone (preferably four microphones) for detecting the direction of sound, and the at least one microphone may be formed as a microphone array.

[0116] Referring to FIG. 3, the control unit (180) of the present invention can detect external sounds at each location of the robot (or at each point located on the path along which the robot moves) while moving the main body of the robot, and determine the direction in which the external sounds are detected. The control unit (180) can record (store) the results of the direction of the external sounds detected at each point (S220).

[0117] The control unit (180) can extract the intersection point of the external sound detection directions determined at the positions (different points) of at least two robots (S230).

[0118] For example, the control unit (180) can detect the intersection of an external sound detection direction (or a corresponding function expression) detected at a first location (or a first point) and an external sound detection direction (or a corresponding function expression) detected at a second location (or a second point) different from the first location.

[0119] Thereafter, the control unit (180) can extract the intersection between the external sound detection direction (or the corresponding function expression) detected at a third location (or third point) different from the first and second locations and the external sound detection direction detected at the first location and the second location, respectively.

[0120] In this case, the control unit (180) can extract the intersection of the external sound detection directions detected at the first position and the second position, the intersection of the external sound detection directions detected at the first position and the third position, and the intersection of the external sound detection directions detected at the second position and the third position.

[0121] The control unit (180) can extract at least one intersection using the external sound detection direction determined at multiple robot locations, and determine a representative value for the at least one intersection using a preset filter.

[0122] The above preset filter may include at least one of a Kalman filter, a particle filter, and DBSCAN (Density-based spatial clustering of application with noise).

[0123] The above preset filters may all be filters that estimate probability-based representative values ​​for intersection points.

[0124] Specifically, the control unit (180) can determine a cluster of a plurality of intersection points of external sound detection directions determined at a plurality of robot locations, and determine the location of the sound source of the external sound based on probability based on the determined cluster (S240).

[0125] Referring to FIG. 3, the control unit (180) stores N pieces of sound data at a certain distance while moving along a given search path. At this time, the control unit (180) measures the average dB (decibels) of N pieces of sound data, and updates N pieces of sound data when the maximum of this value is updated.

[0126] At this time, since the dB size of the sound is an LPF (low pass filtering) signal, short-range noisy sounds can be filtered out and removed.

[0127] The sound data of N obtained in this way may include the robot's position, sound angle, sound elevation, sound raw data, and sound decibel value.

[0128] The control unit (180) uses the 1st to Nth data of the sound data to obtain the equation of the straight line y= ax+b at each point and uses the equation of the straight line for two different robot positions to obtain the intersection point. The number of intersection points obtained in this way is N*(N-1), and the control unit (180) can estimate (assume) that the sound source is within this number.

[0129] The control unit (180) estimates a representative value using at least one of a Kalman filter, a particle filter, and DBSCAN (Density-based spatial clustering of application with noise), which estimate a representative value based on probability, and can estimate that the intersection corresponding to the estimated representative value is the sound source location (sound support location) of the external sound.

[0130] When a robot searches for sound, the closer it gets to the location of the sound, the larger the dB size of the sound data becomes. Ultimately, the location of the sound source is determined when the dB is the largest, so the control unit (180) can obtain the final location of the sound by measuring the location of the sound using triangulation and updating the time when the dB of the sound is the largest.

[0131] Figure 4 (a) is a drawing showing the direction detection result and intersection point according to the movement of the robot, and Figure 4 (b) is a drawing showing only the intersection point according to the movement of the robot.

[0132] The control unit (180) can estimate the direction of a sound source (direction of external sound detection) by performing SSL (Sound Source Localization) based on GCC-PHAT (Generalized Cross Correlation PHAse Transform) using a microphone array composed of four microphones.

[0133] The control unit (180) records the newly detected direction detection results together with the existing direction detection results, stores them in a memory having a total of N result values ​​(N is a natural number), and can detect direction detection intersections using N direction information and the robot's movement path.

[0134] Thereafter, the control unit (180) can estimate a representative value using a probability-based filter (e.g., Kalman filter or particle filter).

[0135] Figure 5 (a) shows a diagram illustrating determining a cluster of intersections using a particle filter and determining the location of an external sound source based on probability based on the determined cluster.

[0136] Figure 5 (b) is a diagram showing estimation of a representative value using DBSCAN (Density-based spatial clustering of applications with noise).

[0137] Using particle filters and DBSCAN (Density-based spatial clustering of applications with noise) requires more computation than the Kalman filter, but can produce the same results.

[0138] Meanwhile, the control unit (180) can utilize the elevation value of the sensor output data of the sensing unit. The elevation value increases as the sensor gets closer to the sound, and the control unit (180) can use the elevation value to determine whether the robot has reached the vicinity of the sound source.

[0139] The robot of the present invention can determine the final location of the sound source by using both dB size and elevation, since the type of sensor changes depending on the shape of the robot.

[0140] Using elevation values ​​can yield more reliable results, but the sensor that outputs elevation values ​​must be equipped with more microphones, which can increase material costs.

[0141] Meanwhile, the present invention can additionally perform search driving to further increase accuracy when the location of the sound source of an external sound is estimated as a probability-based representative value.

[0142] Specifically, when the location of the sound source of the external sound is estimated, the control unit (180) can control the driving unit to perform a search operation to determine the location of the sound source of the external sound based on the estimated location.

[0143] FIG. 6 is a flowchart for explaining a sound source search method according to another embodiment of the present invention, and FIG. 7 is a conceptual diagram for explaining the sound source search method examined in FIG. 6.

[0144] Referring to FIG. 6, the control unit (180) can perform the first mode (State 0) for estimating the sound source location of an external sound based on probability (S602).

[0145] In the above first mode (State 0), as described above, the control unit (180) extracts an intersection point as an accumulated result of the robot's movement path and the direction of the sound source detected during movement while moving the robot, and estimates a representative value using a probability-based filter (S604, S606).

[0146] Specifically, the control unit (180) can detect and cumulatively record the direction of sound source generation using two or more microphone sets, estimate the movement path of the robot in space, and detect the intersection of the sound source direction detection results.

[0147] Thereafter, the control unit (180) estimates the representative value of the detected intersection using a probability estimation-based algorithm (Kalman filter, particle filter, and DBSCAN), and by setting a parameter for distance, if a cluster is formed at a distance greater than a certain distance, it can estimate (infer) the location of the sound source (or the location of a new sound source) (S606).

[0148] When the location of the sound source of an external sound is estimated, the control unit (180) can control the driving unit to perform a search operation to determine the location of the sound source of the external sound based on the estimated location.

[0149] That is, when the location of the sound source of the external sound is estimated, the control unit (180) can enter the second mode (State 1) for performing search driving to determine the location of the sound source of the external sound based on the estimated location (S608).

[0150] The control unit (180) can set (or create) a search point to drive in a predetermined area adjacent to the sound source location centered on the estimated sound source location of the external sound (S610).

[0151] The waypoint driving described in Fig. 6 may mean general driving according to a robot scenario.

[0152] The control unit (180) operates a sound source detection algorithm while driving, and when the sound source direction detection intersection and statistically based representative value are estimated, it converts to a search driving state, selects a certain area based on the sound source location representative value location, and can create a search point that includes the entire perimeter.

[0153] Referring to FIG. 7, the control unit (180) can set (create) a search point (710) around the estimated sound source location (700) of the external sound estimated in the first mode (State 0) (or an adjacent area surrounding the estimated sound source location) (710).

[0154] Specifically, the control unit (180) can drive the robot by setting a driving path that includes the vicinity of the estimated sound source location (700), and can continuously perform a sound source location estimation algorithm during search driving.

[0155] The control unit (180) can exclude places from the path that the robot cannot go.

[0156] Thereafter, the control unit (180) controls the driving unit so that the robot drives while passing through the search point (S612), and can search for the location of the sound source of the external sound while driving while passing through the search point.

[0157] If the control unit (180) determines that the probability of the sound source location of the external sound is greater than a certain value through search point driving, it determines that it is confirmed (S614) and can confirm the sound source location of the external sound.

[0158] Additionally, the control unit (180) can record the location and time of the sound source to determine the location of the sound source of the external sound (S618).

[0159] On the other hand, if the search point has not been determined through the above search point driving, the control unit (180) can determine whether there are any remaining search points (S616), and if there are no search points, the point with the highest probability can be determined as the location of the sound source. In addition, the control unit (180) can store the location and time of the confirmed (or completed search) sound source.

[0160] The control unit (180) can determine the sound source location of the external sound under different conditions in a first mode (State 0) for estimating the sound source location of the external sound, and a second mode (State 1) for searching the sound source location of the external sound while passing through the search point.

[0161] For example, during navigation, if a cluster of intersections satisfies a certain condition, the location of the sound source can be determined.

[0162] The control unit (180) can set the sound source location detection conditions of the first mode (State 0 (normal driving)) and the sound source location detection conditions of the second mode (State 1 (search driving)) for the intersection cluster differently.

[0163] Specifically, the control unit (180) can form a larger number of clusters of multiple intersections of external sound detection directions and a closer cluster distribution in the second mode than in the first mode.

[0164] For example, the control unit (180) may set the probability standard for determining the location of the sound source of an external sound higher in the second mode than in the first mode, and may also set the number of samples of intersection points used to determine the location of the sound source of an external sound higher.

[0165] Afterwards, the control unit (180) can store the sound source location and time when the sound source location of the external sound is confirmed in the second mode.

[0166] In addition, the present invention can perform facility search (or event search) at the initial stage of robot operation through this sound source search method.

[0167] The present invention can record the location and time of a sound by changing the mode (sound search mode) when a sound event occurs during movement of a robot, and the order and activation of the sound search mode can be determined based on the importance of the robot's operation scenario.

[0168] In addition, the present invention can determine the type of sound when a sound event occurs, and if it is determined to be a sound type corresponding to a target sound, it can perform a linked function (for example, a cleaning function if it is determined to be a hair dryer sound) depending on whether there is a history recorded in the memory.

[0169] For example, the present invention can perform a robot function when a record of a target sound type exists, and can record a location and time after performing a sound search mode when there is no record.

[0170] Additionally, the memory of the robot of the present invention can store location, time, type, and function, and perform the corresponding function based on this.

[0171] FIG. 8, FIG. 9, FIG. 10, FIG. 11, FIG. 12, FIG. 13, FIG. 14, FIG. 15, FIG. 16, FIG. 17 and FIG. 18 are conceptual diagrams for explaining a method in which a robot performs a corresponding function based on the location, time and type of sound after searching for the source of sound according to the present invention.

[0172] Referring to FIG. 8, the robot of the present invention may include a memory (170) (or long-term memory (920), hereinafter referred to as memory (920)).

[0173] The memory (920) may store information including at least one of the location of the sound source of an external sound, the time at which the external sound was received (or generated), the type of the external sound, and the function performed by the robot when the external sound was generated (or information linked to the location, time, type, and function performed by the robot).

[0174] When a sound event occurs, the control unit (180) can recognize the sound (or type of sound), detect the direction of the sound, detect the location of the sound based on map information, and determine the function to be performed by the robot based on the time information at which the sound event occurred. The function to be performed by the robot can be determined in advance based on the sound location, time information, and type of sound.

[0175] Thereafter, the control unit (180) can perform a function that the robot must perform in response to the generated sound. At this time, the control unit (180) can perform a function corresponding to the generated external sound based on information including at least one of the location of the sound source of the external sound stored in the memory (920), the time at which the external sound was received (or generated), the type of the external sound, and the function performed by the robot when the external sound was generated.

[0176] The robot of the present invention includes a memory (920), and the control unit (180) can store information including at least one of the estimated sound source location of the external sound, the time at which the external sound was received, the type of the external sound, and the function performed by the robot in the memory (920).

[0177] The present invention can record the location, type and time of sounds detected by a robot while moving in a memory (long-term managed memory) (920), and provide a service by analyzing the memory (920) for the detected sounds.

[0178] At this time, the method of estimating the location of the sound in the control unit (180) is to estimate the direction of the sound using an algorithm such as GCC-PHAT-based Sound Source Localization, extract the intersection of the estimated directions during robot movement, and use the representative value of the intersection as the location of the sound.

[0179] The control unit (180) can estimate the type of sound using an algorithm such as deep learning-based ResNet.

[0180] The control unit (180) can display (store) the location of sound as x, y coordinates based on a map generated using the robot's SLAM.

[0181] Memory (920) (Long-term memory) can record (store) the location of the sound, the time of occurrence, the type, and the function performed by the robot.

[0182] The control unit (180) compares the direction, type, and time of sounds detected during robot operation with the information stored in the memory based on the data recorded in the memory (920) to estimate a location with high similarity and perform movement and a function, or if there is a location or type of sound that is repeated at the same time, it can move to that location and perform a function before the sound is generated.

[0183] The control unit (180) can perform sound type recognition and sound direction detection when the sound event response function of the robot is activated.

[0184] In addition, the control unit (180) can store the location within the map, the sound type recognition result, and the time information in the memory (920) through sound location detection when a sound event occurs, and can also store the function performed by the robot according to the scenario in the memory.

[0185] The control unit (180) can determine and perform an appropriate function among predefined robot functions by referring to information stored in the memory when a sound event occurs.

[0186] Referring to Fig. 9, when the control unit (180) enters state 0 mode and a sound is input (S904), it can compare the importance of the function currently being performed by the robot to determine whether the importance of sound search is possible (S906, S908).

[0187] The control unit (180) compares the importance of the function of the robot currently performing and the importance of the function of the robot performing sound direction / location recognition when a sound event occurs, and if the importance is higher when the sound event occurs, it determines that sound search is possible and can perform sound direction and location recognition (S910).

[0188] The control unit (180) determines whether an event has occurred, and if it is determined that an event has occurred, it can enter the next mode, State 1 mode (S912, S914).

[0189] When the control unit (180) enters state 1 mode, it can recognize the type of sound (S916).

[0190] For example, the control unit (180) can recognize the type of sound using an algorithm such as deep learning-based ResNet.

[0191] The control unit (180) can determine whether the type of recognized sound corresponds to a preset type (or target sound) (S918).

[0192] The control unit (180) can determine whether there is corresponding information among the information stored in the memory (920) if the type of the recognized sound corresponds to a preset type, i.e., a target sound (S920).

[0193] That is, when an external sound is received, the control unit (180) can recognize the type of the external sound, and if the type of the external sound corresponds to a preset type, it can determine whether corresponding information exists among the information stored in the memory.

[0194] The control unit (180) enters state 2 mode when corresponding information exists (S922), and can perform a function using information about the sound location, the time at which the sound was generated, and the function performed by the robot included in the corresponding information (S924).

[0195] Thereafter, the control unit (180) may enter the next mode, state 3 mode, if information corresponding to the recognized sound type does not exist in the memory or if recognition of the exact location of the sound is required (for example, if the recognized sound type is a type of sound that requires precise cleaning, such as the sound of a hair dryer or the sound of breaking a glass).

[0196] When entering State 3 mode, the control unit (180) can create a search driving point (S928) and perform search driving (S930) to recognize the exact location of the sound.

[0197] Thereafter, if the location is not confirmed (S932), the control unit (180) can perform sound location search by driving all remaining search points (S936), and if the location is confirmed, the sound location and time can be stored in the memory (920) (S934).

[0198] That is, in State 0, the control unit (180) can perform a function according to the robot's scenario and determine whether to recognize the sound direction / location according to the importance of the scenario.

[0199] When a sound event occurs (when a sound position representative value is detected), the control unit (180) can be converted to State 1.

[0200] In State 1, the control unit (180) can recognize the type of sound and search the memory (920) if the type of sound matches the target sound.

[0201] The control unit (180) can perform the corresponding robot function if there is a similar record in the memory, and can switch to State 3 if there is no similar record in the memory.

[0202] The control unit (180) can perform a search operation around the initially searched sound location to determine the exact location in State 3 and record the sound location, type, and time in the memory (920).

[0203] Meanwhile, if the corresponding information exists in the memory, the control unit (180) may use the corresponding information to move the robot to a location estimated as the source location of the external sound, and enter a mode (State 3) for searching for the source location of the external sound while passing through a search point based on the estimated location.

[0204] That is, the control unit (180) of the present invention, even when corresponding information exists, enters state 3 mode to set search points around the estimated sound location in order to search for a more accurate sound location, and can more accurately specify the sound location while passing through the search points.

[0205] When the location of the sound source of the external sound is detected, the control unit (180) can update the memory (920) with information including at least one of the location of the sound source of the external sound, the time at which the external sound was received, the type of the external sound, and the function performed by the robot.

[0206] The control unit (180), as illustrated in FIG. 10, can store information (1000) including at least one of the location of the sound source of an external sound, the time at which the external sound was received, the type of the external sound, and the function performed by the robot.

[0207] The location of the sound source of the external sound, the time at which the external sound was received, the type of the external sound, and the function performed by the robot may be stored in association with each event (or each piece of information).

[0208] The above memory (920) may store information about the location of sound generation (x, y coordinates on the map), type of sound, time of sound generation (time of sound reception), and function performed by the robot.

[0209] Additionally, the memory (920) may further store information on the displacement of sound generation time, sound maintenance time, and displacement of sound maintenance time.

[0210] When a sound (external sound) is received, the control unit (180) can search (search) for sound occurrence records such as a record of sound occurrence within a certain area of ​​the detected sound occurrence location, a record of sound occurrence within a certain range of the detected sound time, and a type of detected sound, based on the information stored in the memory (920) described above.

[0211] FIGS. 11 to 18 are conceptual diagrams for explaining various embodiments in which a robot of the present invention performs a function related to a sound event when detecting a sound event.

[0212] First, referring to FIG. 11, the control unit (180) can obtain information about the space in which the robot can drive through map information stored in memory or a map generation function.

[0213] The control unit (180) can move the robot to explore the facility, check / update map information, and estimate the location of the sound source from which the sound was generated.

[0214] The control unit (180) can transmit information acquired through the communication unit to a monitoring device (monitoring device, PC, terminal, server, etc.). The user can receive initial facility search information and sound source location estimation information searched by the robot, output to the monitoring device, and can set a management point (e.g., a place where the robot will intensively perform its function) or be provided with a location (recommended point) where the robot will perform its function based on the sound location.

[0215] Referring to FIG. 12, the robot of the present invention can be used for facility management, and can detect a facility corresponding to the location of a sound source through vision recognition to increase accuracy.

[0216] For example, as shown in (a) of Fig. 12, when a new sound source is found in a new location, the control unit (180) can estimate (confirm) the location of the sound source according to the method for finding the location of the external sound source described above, and newly store information on the location of the sound, the time of occurrence, and the type of the sound in the memory.

[0217] As illustrated in FIG. 12(b), when at least one sound source among the previously stored (searched) sound sources disappears, the control unit (180) can transmit information notifying that the sound source has disappeared to a monitoring device.

[0218] As illustrated in Fig. 12(c), even when the type of sound among the previously stored (searched) sound sources has changed, the control unit (180) can transmit information notifying that the type of sound source has changed to a monitoring device.

[0219] In this way, the present invention can improve customer convenience by setting (selecting) a management point through sound, managing even parts not detected by vision, and providing a response method through monitoring when an abnormal situation occurs.

[0220] Referring to (a), (b), and (c) of FIG. 13, the control unit (180) can link the sound generation location and the type of sound generation by time zone and store them in memory. The control unit (180) can map the sound generation location and the type of sound generation by time zone to map information to create a semantic map.

[0221] Thereafter, as shown in (a), (b), (c), and (d) of FIG. 14, when a user makes a sound calling the robot at a specific time (e.g., 9 p.m.) from a place where the robot is not visible, the control unit (180) can move to the location where the sound was generated at the corresponding time based on the location of sound generation and the type of sound generation stored in the memory.

[0222] Thereafter, the control unit (180) can perform user recognition using a vision sensor to improve accuracy by additionally confirming sound events in addition to the location of the sound, the time of sound occurrence, and the type of sound.

[0223] Through this, the present invention generates a semantic map including time through acoustic location and type recognition through sound (supplemented by vision and multimodal recognition), and can estimate the location of the user who called the robot based on the semantic map to quickly respond.

[0224] Likewise, referring to (a), (b), and (c) of FIG. 15, the control unit (180) can link the sound generation location and the type of sound generation by time zone and store them in memory. The control unit (180) can map the sound generation location and the type of sound generation by time zone to map information to create a semantic map.

[0225] Referring to (a), (b), (c) and (d) of FIG. 16, the control unit (180) can detect changes in a space (e.g., a home) where the robot exists based on this semantic map and provide a function to recall memories on a specific day (e.g., an anniversary or a point in time after a certain period of time has elapsed).

[0226] For example, when a new sound is generated, as shown in (a) and (b) of FIG. 16, the control unit (180) can perform a function (e.g., a shooting function for a new subject) linked to the type of sound based on the location where the sound was generated and the type of the sound, as shown in (c) of FIG. 16.

[0227] Thereafter, the control unit (180) can transmit the image captured by the above function to a linked monitoring device (or the user's mobile terminal) after a predetermined period of time has elapsed.

[0228] Through this, the present invention can provide a service that recognizes environmental changes through sound events and recalls memories based on the date of change.

[0229] In addition, referring to (a), (b), and (c) of FIG. 17, the control unit (180) can link the sound generation location and the sound generation type by time zone and store them in memory. The control unit (180) can map the sound generation location and the sound generation type by time zone to map information to create a semantic map.

[0230] The present invention can improve satisfaction with safety by detecting a dangerous situation in a space where a robot exists (e.g., a home) and notifying the user.

[0231] Referring to (a), (b), (c) and (d) of FIG. 18, when an external sound is received, the control unit (180) can estimate and search for the location where the sound was generated, recognize the type of sound, and perform a function linked to the type of the recognized sound.

[0232] For example, as shown in (c) of FIG. 18, if the type of sound is the sound of glass breaking, the control unit (180) can capture a portion corresponding to the location of the sound through a vision sensor (camera) equipped on the robot, and, as shown in (d) of FIG. 18, transmit the captured image to a preset device (monitoring device, user's mobile terminal, etc.).

[0233] Through this configuration, the present invention can perform sound-based hazard detection triggering, identify the location of a situation through a sound-generated event by utilizing information stored in memory, and notify the user of a hazard situation using multi-modality, thereby increasing the hazard situation recognition rate.

[0234] According to at least one of the embodiments of the present invention, the present invention can provide a method in which a robot can estimate a sound source while moving, and perform another search drive by driving around the estimated sound source for accurate judgment, thereby improving the accuracy of the sound source location.

[0235] According to at least one of the embodiments of the present invention, the present invention can provide a method in which a robot can estimate a sound source while moving, and perform another search drive by driving around the estimated sound source for accurate judgment, thereby improving the accuracy of the sound source location.

[0236] The present invention described above can be implemented as a computer-readable code on a medium in which a program is recorded. The computer-readable medium includes all types of recording devices that store data that can be read by a computer system. Examples of the computer-readable medium include a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc., and also includes media implemented in the form of a carrier wave (e.g., transmission via the Internet). In addition, the computer may include a control unit (180) of a robot (10) according to an embodiment of the present invention.

[0237] Accordingly, the above detailed description should not be construed as limiting in all respects, but rather as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are intended to be included within the scope of the present invention.

Claims

1. A driving unit formed to move the main body of the robot; a microphone that receives external sounds; and A robot including a control unit that determines the direction in which external sound is detected for each position of the robot while moving the main body of the robot, extracts the intersection of the external sound detection directions determined from the positions of at least two robots, and estimates the location of the sound source of the external sound using at least one of the extracted intersections.

2. In paragraph 1, The above control unit, Extracting at least one intersection point using the external sound detection direction determined from multiple robot locations, A robot characterized in that it determines a representative value for at least one intersection using a preset filter.

3. In paragraph 2, A robot characterized in that the above preset filter includes at least one of a Kalman filter, a particle filter, and DBSCAN (Density-based spatial clustering of application with noise).

4. In paragraph 1, The above control unit, A robot characterized in that it determines a cluster of a plurality of intersection points of external sound detection directions determined at a plurality of robot locations, and determines the location of the sound source of the external sound based on probability based on the determined cluster.

5. In paragraph 1, The above control unit, A robot characterized in that, when the location of the sound source of the external sound is estimated, the driving unit is controlled to perform a search operation to determine the location of the sound source of the external sound based on the estimated location.

6. In paragraph 1, The above control unit, A robot characterized in that it sets a search point to drive in a predetermined area adjacent to the sound source location centered on the estimated sound source location of the external sound.

7. In paragraph 6, The above control unit, Control the driving unit so that the robot moves while passing through the search point, A robot characterized in that it searches for the location of the sound source of the external sound while driving via the above search point.

8. In paragraph 7, The above control unit, A robot characterized in that the location of the sound source of the external sound is determined under different conditions in a first mode for estimating the location of the sound source of the external sound and a second mode for searching the location of the sound source of the external sound while passing through the search point.

9. In paragraph 8, The above control unit, A robot characterized in that, in the second mode, the number of clusters of multiple intersections of external sound detection directions is greater than in the first mode, and a closer cluster distribution is formed.

10. In paragraph 7, The above control unit, A robot characterized in that, when driving to the above search point is completed, the location of the sound source of the external sound is determined.

11. In paragraph 1, Includes more memory, The above control unit, A robot characterized in that it stores information including at least one of the estimated sound source location of the external sound, the time at which the external sound was received, the type of the external sound, and the function performed by the robot in the memory.

12. In paragraph 11, The above control unit, A robot characterized in that, when an external sound is received, the type of the external sound is recognized, and if the type of the external sound corresponds to a preset type, it determines whether corresponding information exists among the information stored in the memory.

13. In paragraph 12, The above control unit, If the corresponding information exists in the memory, the robot is moved to a location estimated to be the source of the external sound using the corresponding information, A robot characterized in that it enters a mode for searching for the location of an external sound source while passing through a search point based on the estimated location.

14. In paragraph 13, The above control unit, A robot characterized in that, when the location of the sound source of the external sound is detected, information including at least one of the location of the sound source of the external sound, the time at which the external sound was received, the type of the external sound, and the function performed by the robot is updated in the memory.

Images