Robot, and acoustic source detection method of robot

Abstract

A robot according to an embodiment of the present invention comprises: a driving unit for moving a main body of the robot; a microphone for receiving external sound; and a control unit which extracts rooms and doors by using a map corresponding to the space in which the robot is present, extracts a Voronoi node for each room from the map, uses a region of interest set by a user and the Voronoi node extracted for each room, so as to determine the order in which the robot passes, such that a preset condition is satisfied, and controls the driving unit such that the robot passes through the region of interest and / or the Voronoi node according to the order in order to find the source of the external sound, wherein, if it is determined that the direction of the external sound is toward the outside of the room corresponding to the door when the robot passes through the room, the control unit causes the robot to skip passing through the region of interest and / or the Voronoi node present in the room corresponding to the door.

Description

Robots and methods for detecting the source of sound from robots

[0001] The present invention relates to a robot and a method for detecting the source of sound of the robot.

[0002] Robots were developed for industrial use and have played a role in factory automation. Recently, the scope of robotics applications has expanded, with developments in medical and aerospace robots, as well as domestic robots for general use. Among these robots, those capable of autonomous navigation are called mobile robots. A representative example of a mobile robot used in the home is a robot vacuum cleaner.

[0003] Several technologies are known for detecting the environment and user surrounding a robot vacuum cleaner through various sensors installed in the robot vacuum cleaner. Furthermore, technologies are known for allowing a robot vacuum cleaner to learn and map its driving area and determine its current location on the map. Robot vacuum cleaners are known for cleaning their driving area in a preset manner.

[0004] Currently, with the advancement of artificial intelligence technology, it is being applied in various fields. Following this trend, deep learning technology is also being applied in the field of acoustic recognition. Robots (or acoustic recognition devices) are emerging that detect various acoustic signals, or sound events, using various deep learning models trained with augmented data from diverse acoustic environments.

[0005] Meanwhile, a typical deep learning model has a configuration that produces probability vectors representing the probability that a received acoustic signal is a specific acoustic signal for each of a plurality of different specific acoustic signals. Accordingly, acoustic recognition devices to which the typical deep learning model is applied (hereinafter, a typical acoustic recognition device) determine, based on thresholding, one of the probability vectors for each of a plurality of different acoustic signals produced in response to the received acoustic signal, and recognize the received acoustic signal as a specific acoustic signal corresponding to the determined probability vector. Then, based on the recognition result of the acoustic signal, it determines whether an event has occurred and performs an action accordingly.

[0006] In this way, conventional acoustic recognition devices recognize acoustic signals based on a boundary value and one of the probability vectors generated from the applied deep learning model, so the accuracy of acoustic recognition can be determined depending on the determination of the boundary value. Therefore, the conventional acoustic recognition devices are configured to generate probability vectors for each acoustic signal multiple times, i.e., across multiple frames, based on the received acoustic signal in order to generate an appropriate boundary value, and to calculate the average value of the probability vectors generated for each frame. Therefore, conventional acoustic recognition devices have a problem in that it takes time to correspond to several frames for acoustic recognition, which hinders real-time acoustic recognition.

[0007] One object of the present invention is to provide a robot capable of searching for a sound source in an optimized manner and a control method thereof.

[0008] Another object of the present invention is to provide a robot and a control method thereof that can accurately detect a sound source while reducing the search time.

[0009] According to an embodiment of the present invention for achieving the above or other purposes, a robot includes a driving unit configured to move a main body of the robot, a microphone configured to receive external sounds, and a control unit configured to extract rooms and doors using a map corresponding to a space in which the robot exists, extract a Bornoi node for each room from the map, determine an order in which the robot will pass through the regions of interest set by a user and the Bornoi nodes extracted for each room so as to satisfy preset conditions, and control the driving unit so that the robot passes through at least one of the regions of interest and the Bornoi nodes according to the order in order to find the source of the external sound, wherein the control unit is characterized in that, when the robot passes through the door, if the direction of the external sound is determined to be outside the room corresponding to the door, the robot does not pass through at least one of the regions of interest and the Bornoi nodes existing in the room corresponding to the door.

[0010] In an embodiment, the control unit is characterized in that it obtains a Bornoi graph based on a map of a space in which the robot moves, and extracts a Bornoi node that satisfies a preset condition from the Bornoi graph.

[0011] In an embodiment, the control unit is characterized in that it performs distance transformation on the map to obtain a new map in which a larger value is expressed as the distance from the wall increases, and obtains a Bornoi graph by tracking a change in pixel value in the new map in a direction within a preset range.

[0012] In an embodiment, the control unit is characterized in that it sets a portion where graphs intersect in the Bornoi graph as a Bornoi node, and sets the Bornoi node so that at least one Bornoi node is included in each room.

[0013] In an embodiment, the control unit is characterized in that it determines whether an area of ​​interest set by the user exists, and if an area of ​​interest exists, it determines the order to first pass through the area of ​​interest closest to the robot.

[0014] In an embodiment, the control unit is characterized in that, when a plurality of regions of interest are set, the order is determined to pass through the plurality of regions of interest by the shortest distance and then pass through the Bornoi node.

[0015] In an embodiment, the control unit is characterized in that it determines the order to go through the closest region of interest and then go through the closest region of interest or Bornoi node from the closest region of interest.

[0016] In an embodiment, the control unit is characterized in that it determines whether an area of ​​interest set by the user exists, and if the area of ​​interest does not exist, it determines the order to first pass through the Bornoi node closest to the robot.

[0017] In an embodiment, the control unit is characterized in that it determines the order to go through the nearest Bornoi node and then go through the nearest Bornoi node from the nearest Bornoi node.

[0018] In an embodiment, the control unit is characterized in that it determines the direction of an external sound received through the microphone, and when the robot arrives at the location of the visit, it determines whether the direction from which the external sound is heard is inside or outside the room corresponding to the visit.

[0019] In an embodiment, the control unit is characterized in that, when the direction from which the external sound is heard is outside the room corresponding to the visit, the control unit controls the driving unit so that the robot does not pass through a region of interest or a Bornoi node existing in the room corresponding to the visit.

[0020] In an embodiment, the control unit is characterized in that, when it is determined that the external sound is received from outside the room corresponding to the visit, it processes it as having passed through the area of ​​interest or the Bornoi node existing in the room corresponding to the visit, even if it does not pass through the area of ​​interest or the Bornoi node existing in the room corresponding to the visit.

[0021] In an embodiment, the control unit is characterized in that it performs a sound location estimation algorithm for determining the source of the external sound while moving the robot.

[0022] In an embodiment, the control unit is characterized in that, when the route to the region of interest set on the map and the Bornoi node are both completed, the control unit controls the driving unit to move the robot to the initial position to which it moved.

[0023] The effects of the acoustic recognition device and the control method thereof according to the present invention are described as follows.

[0024] According to at least one of the embodiments of the present invention, the present invention can search all rooms using a region of interest to reduce search errors and increase the success rate of finding a sound source.

[0025] The present invention can significantly reduce search time by using the visit information of each room to move to another room without searching the room if the direction of the sound is outside the room.

[0026] The present invention can enhance noise vulnerability by obtaining an envelope curve of raw sound data in the process of detecting a sound location and determining the sound source using the maximum and minimum values ​​of the envelope curve.

[0027] Figure 1 is a block diagram illustrating the configuration of a robot according to an embodiment of the present invention.

[0028] Figure 2 is a conceptual diagram to explain why it is difficult to determine the source of a sound when using a conventional method.

[0029] Figure 3 is a flowchart for explaining a representative sound source search method of the present invention.

[0030] Figures 4, 5, 6, 7, 8, 9, 10 and 11 are conceptual diagrams for explaining the sound source search method examined in Figure 3.

[0031] FIG. 12 and FIG. 13 are flowcharts for explaining a method for searching for a sound source and a method for estimating a sound location of a robot according to one embodiment of the present invention.

[0032] It should be noted that the technical terms used herein are used merely to describe specific embodiments and are not intended to limit the present invention. Furthermore, singular expressions used herein include plural expressions unless the context clearly dictates otherwise. The suffixes "module" and "part" used in the following description for components are assigned or used interchangeably solely for the convenience of writing the specification, and do not in themselves have distinct meanings or roles.

[0033] In this specification, the terms “comprises” or “includes” should not be construed to necessarily include all of the components or steps described in the specification, and some of the components or steps may not be included, or additional components or steps may be included.

[0034] In addition, when describing the technology disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the technology disclosed in this specification, the detailed description is omitted.

[0035] In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention. In addition, not only each embodiment described below, but also a combination of embodiments may correspond to the spirit and technical scope of the present invention as modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention.

[0036] Figure 1 is a block diagram illustrating the configuration of a robot according to an embodiment of the present invention.

[0037] A robot (or, acoustic recognition device) (10) according to an embodiment of the present invention may be configured to include a control unit (180), a communication unit (110), an input unit (120), a driving unit (130), a sensing unit (140), an output unit (150), a memory (170), and an interface unit (160) connected to the control unit (180). The components illustrated in FIG. 1 are not essential for implementing the robot (10), and thus, the robot (10) described in this specification may have more or fewer components than the components listed above.

[0038] More specifically, among the above components, the communication unit (110) may include one or more modules that enable wireless communication between the robot (10) and a wireless communication system, between the robot (10) and at least one peripheral device, or between the robot (10) and an external server.

[0039] This communication unit (110) may include at least one of a wireless Internet module (111), a short-range communication module (112), and a location information module (113).

[0040] The wireless Internet module (111) refers to a module for wireless Internet access, and can be built into or externally mounted on the robot (10). The wireless Internet module (111) is configured to transmit and receive wireless signals in a communication network according to wireless Internet technologies.

[0041] The short-range communication module (112) is for short-range communication and can support short-range communication using at least one of Bluetooth, RFID, infrared communication, UWB, ZigBee, NFC, Wi-Fi, Wi-Fi Direct, and Wireless USB technologies. This short-range communication module (112) can support wireless communication between the robot (10) and a wireless communication system, between the robot (10) and a peripheral device, or between the robot (10) and a network where an external server is located through a short-range wireless communication network (Wireless Area Network).

[0042] The location information module (113) is a module for obtaining the location (or current location) of the robot (10), and representative examples thereof include a GPS (Global Positioning System) module or a WiFi (Wireless Fidelity) module. For example, if the robot (10) utilizes a GPS module, it can obtain the location of the robot (10) using signals transmitted from GPS satellites. As another example, if the robot (10) utilizes a Wi-Fi module, it can obtain the location of the robot (10) based on information from a wireless AP (Wireless Access Point) that transmits or receives wireless signals with the Wi-Fi module.

[0043] As needed, the location information module (113) may perform the function of any of the other modules of the communication unit (110) to obtain data regarding the location of the robot (10) as a substitute or additionally. The location information module (113) is a module used to obtain the location (or current location) of the robot (10), and is not limited to a module that directly calculates or obtains the location of the robot (10).

[0044] The input unit (120) may include a camera (121) or a video input unit for inputting a video signal, a microphone (122, hereinafter referred to as a microphone) or an audio input unit for inputting an audio signal, and a user input unit (123, for example, a touch key, a mechanical key, etc.) for receiving information from a user. Voice data or image data collected by the input unit (120) may be analyzed and processed into a user's control command.

[0045] Meanwhile, the camera (121) and microphone (122) can each collect video and audio signals from around the robot (10). In this case, the camera (121) may be equipped with one or more image sensors and can process image frames, such as still images or moving images, obtained by the image sensors. The processed image frames can then be stored in the memory (170).

[0046] In addition, the microphone (122) can sample external acoustic signals at preset time intervals and process them into acoustic data. The processed acoustic data can be converted into data for acoustic recognition in the robot (10), for example, a frame-by-frame mel spectrogram, and the converted mel spectrogram can be used as an input for a deep learning model for acoustic recognition. Meanwhile, the microphone (122) can be implemented with various noise removal algorithms to remove noise generated in the process of receiving an external acoustic signal. The microphone (122) can be designed with a beamforming structure having directionality, and can be formed so as to further improve the reception rate of acoustic information in a specific direction. In this case, the specific direction can be the front of the robot (10).

[0047] Meanwhile, the driving unit (130) may be a component for performing the function of the robot (10) or the device equipped with the robot (10).

[0048] The above driving unit (130) may be a driving unit for performing a specific function of the robot or moving the main body of the robot.

[0049] For example, if the robot (10) is a robot vacuum cleaner that performs a cleaning function, the driving unit (130) may include at least one component for performing the cleaning function. In this case, the driving unit (130) may include a suction unit for vacuuming dust. In addition, the driving unit (130) may include at least one wheel for movement, at least one motor for controlling the rotation of the wheel, and at least one actuator. In addition, the driving unit (130) may include a brush, and at least one motor for controlling the rotation of the brush.

[0050] Meanwhile, if the robot (10) is a robot for performing a patrol function, the driving unit (130) may include a component for the patrol function. For example, the driving unit (130) may include a moving unit (e.g., a wheel, a motor, an actuator) for moving the robot (10), and may include a lighting unit including at least one light for supporting the patrol function.

[0051] Meanwhile, if the robot (10) is formed into a robot that performs a specific function, the driving unit (130) may be the robot itself. In this case, the control unit (180) may be the control unit of the robot, or the control unit of the robot may function as the control unit of the robot (10).

[0052] The above driving unit (130) is equipped with a wheel unit for driving the robot (10). The robot (10) can be moved forward, backward, left, and right or rotated by the wheel unit.

[0053] The wheel unit includes a main wheel and a sub wheel.

[0054] The main wheels are provided on each side of the robot (10) and are configured to rotate in one or the other direction according to a control signal from the control unit. Each main wheel may be configured to be driven independently of the other. For example, each main wheel may be driven by a different motor. Alternatively, each main wheel may be driven by multiple different axes provided on a single motor.

[0055] The sub-wheel supports the robot (10) together with the main wheel and is configured to assist the driving of the robot (10) by the main wheel.

[0056] The control unit controls the operation of the wheel unit, thereby enabling the robot (10) to autonomously drive on the floor.

[0057] Meanwhile, the robot (10) is equipped with a battery (not shown) that supplies power to the robot (10). The battery is configured to be rechargeable and can be configured to be detachably attached to the bottom of the robot (10).

[0058] The sensing unit (140) may include one or more sensors for sensing at least one of information within the robot (10) and information about the surrounding environment surrounding the robot (10). For example, the sensing unit (140) may include at least one of a proximity sensor (141), an illumination sensor (142), a touch sensor, an acceleration sensor, a magnetic sensor, a gravity sensor (G-sensor), a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor (IR sensor), an ultrasonic sensor, an optical sensor (e.g., a camera (see 121)), a microphone (see 122), an environmental sensor (e.g., a barometer, a hygrometer, a thermometer, a radiation detection sensor, a heat detection sensor, a gas detection sensor, etc.), a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric sensor, etc.), and a sound sensor (including at least one microphone). Meanwhile, the robot (10) disclosed in this specification can utilize information sensed by at least two of these sensors in combination.

[0059] The output unit (150) is for generating output related to visual, auditory or tactile sensations, and may include a display unit (151) and an audio output unit (152).

[0060] The display unit (151) can display (output) various information (hereinafter referred to as image information) related to the operation of the robot (10) and information related to the operation of the driving unit (130). For example, the display unit (151) can display information on a recognized sound signal, an execution screen of a currently executing function, or UI (User Interface) or GUI (Graphic User Interface) information according to the displayed information.

[0061] In addition, the audio output unit (152) can output various audio data according to the control of the control unit (180). For example, the audio output unit (152) can output information about a recognized audio signal in the form of a human voice or output audio data related to a currently executing function (e.g., voice alarm, siren, etc.) according to the control of the control unit (180).

[0062] The interface unit (160) serves as a passageway for various types of external devices connected to the robot (10). The interface unit (160) may include at least one of an external charger port connected to an external charger, a wired / wireless data port, an audio output port, a video output port, and an earphone port.

[0063] Meanwhile, the normalization unit (not shown) can generate a Mel spectrum vector by collecting sound data generated by sampling an acoustic signal from a microphone (122), and can generate a Mel spectrogram by collecting the generated Mel spectrum vector. Here, Mel is a subjective perception amount of an acoustic signal according to a frequency recognizable to a human, and can be a unit expressed linearly by tuning a nonlinear frequency unit according to a frequency recognizable to a human. In other words, Mel can mean a value converted through a frequency logarithmic function (Mel scale function).

[0064] That is, a Mel spectrogram (or Mel scale spectrogram) is a spectrogram that reflects the human audible frequency range by dividing the frequency of an audio signal into a preset number of Mel units and expressing it, and can represent the spectral change of an audio signal over time.

[0065] And the normalization unit can detect the mel spectrum vector having the maximum value among the mel spectrum vectors constituting the generated mel spectrogram. And the mel spectrogram can be normalized based on the detected maximum value. For example, the normalization unit (180) can normalize the mel spectrogram by dividing the mel spectrogram by the square of the detected maximum value. And the mel spectrogram normalized by the normalization unit can be input to the artificial intelligence unit for the recognition of the acoustic signal as feature information for the recognition of the acoustic signal.

[0066] Meanwhile, the artificial intelligence department (not shown) performs the role of processing information based on artificial intelligence technology, and may include one or more modules that perform at least one of information learning, information inference, information perception, and natural language processing.

[0067] The above artificial intelligence unit can use machine learning technology to determine what kind of acoustic signal the data (hereinafter referred to as acoustic data) sampled from an acoustic signal received through a microphone (122) is. To this end, the artificial intelligence unit can infer a probability vector for each of a plurality of different acoustic signals set in advance for the acoustic data based on a deep learning model based on information learned using the machine learning technology.

[0068] Here, learning can be achieved through the aforementioned machine learning technology. This machine learning technology, based on at least one algorithm, collects and learns large amounts of information, and uses the learned information to make judgments and predictions. Information learning involves identifying the characteristics, rules, and judgment criteria of the information, quantifying the relationships between information, and using these quantified patterns to predict new data.

[0069] Meanwhile, the probability vector inferred through the deep learning model may be a probability value indicating the probability that the acoustic data is a specific acoustic signal. In other words, the probability vectors for each of the multiple different acoustic signals inferred based on the deep learning model may indicate the probabilities that the received acoustic signal may be an acoustic signal corresponding to each of the multiple different acoustic signals.

[0070] Here, the above-mentioned deep learning technology utilizes an artificial neural network (ANN) model to perform at least one of learning, judging, and processing of information. The ANN may have a structure that connects layers (hidden layers) and transmits data between layers.

[0071] Meanwhile, the artificial intelligence unit may use a convolutional neural network (CNN) model to infer probability vectors of each of a plurality of different preset acoustic signals for the acoustic data. Furthermore, the convolutional neural network model (CNN) may be a model trained with augmented data from various acoustic environments, for example, at least one of which is different from the SNR (Signal to Noise) and RT 60 (Reverberation Time 60).

[0072] In addition, the artificial intelligence unit may further include an acoustic recognition decision model capable of determining a received acoustic signal as a specific acoustic signal based on probability vectors corresponding to each of a plurality of different acoustic signals inferred by the deep learning model, i.e., class-specific probability vectors (a plurality of different acoustic signals constitute a plurality of classes). Here, the acoustic recognition decision model may be a model according to the machine learning algorithm, and may be a model learned based on the class-specific probability vectors.

[0073] Meanwhile, the memory (170) can store data supporting the functions of the robot (10). The memory (170) can store a plurality of application programs (or applications) running on the robot (10), data for the operation of the robot (10), commands, data for the operation of the artificial intelligence unit (190) (e.g., hidden layers of a deep learning model or a machine learning model and weight information of each hidden layer, information for updating or learning, etc.).

[0074] In addition, the memory (170) may store data obtained by sampling the sound signal received from the microphone (122), i.e., sound data. In addition, based on the stored sound data, a Mel Spectrogram generated by the normalization unit (180), data required for normalization of the Mel Spectrogram, and information on the Mel Spectrogram normalized by the normalization unit may be stored.

[0075] Meanwhile, the control unit (180) controls each connected component and can control the overall operation of the robot (10).

[0076] For example, the control unit (180) can generate sound data by sampling an acoustic signal received through a microphone (122).

[0077] The control unit (180) can determine that an event has not occurred if the received acoustic signal is the noise signal. In this case, the control unit (180) can control the driving unit (130) not to perform a function corresponding to the occurrence of the event. Accordingly, the control unit (180) can control the driving unit (130) to continue performing the currently performing function or to maintain the current operating state.

[0078] Meanwhile, the type of the robot (10) or the robot (10) may be determined depending on the function performed by the robot (10) or the robot (10).

[0079] Meanwhile, the control unit (180) typically controls the overall operation of the robot (10). The control unit (180) can provide or process functions related to sound recognition by processing signals, data, information, etc. input or output through the components discussed above or by operating an application program stored in the memory (170).

[0080] In addition, the control unit (180) can control at least some of the components discussed in FIG. 1 to drive an application program stored in the memory (170). Furthermore, the control unit (180) can operate at least two or more of the components included in the robot (10) in combination to drive the application program.

[0081] A robot (10) according to one embodiment of the present invention may be a robot vacuum cleaner that performs cleaning. Accordingly, the robot (10) of the present invention may include a cleaning unit (not shown) for performing cleaning.

[0082] The cleaning unit is positioned in a protruding form from one side of the main body of the vacuum cleaner, and can suck in air containing dust or perform mopping. The one side can be the side on which the main body of the vacuum cleaner moves in the forward direction (F), i.e., the front of the main body of the vacuum cleaner.

[0083] This drawing shows that the cleaning unit has a shape that protrudes forward and to both the left and right sides from one side of the main body of the cleaner. Specifically, the front end of the cleaning unit is positioned at a position spaced forward from one side of the main body of the cleaner, and the left and right ends of the cleaning unit are positioned at positions spaced left and right from one side of the main body of the cleaner, respectively.

[0084] Since the main body of the vacuum cleaner is formed in a circular shape and the rear ends of the cleaning unit are formed to protrude from the main body of the vacuum cleaner to the left and right, an empty space, i.e., a gap, can be formed between the main body of the vacuum cleaner and the cleaning unit. The empty space is a space between the left and right ends of the main body of the vacuum cleaner and the left and right ends of the cleaning unit, and has a recessed shape toward the inside of the robot (10).

[0085] If an obstacle is caught in the above-mentioned empty space, the robot (10) may become stuck on the obstacle and become unable to move. To prevent this, a cover member may be arranged to cover at least a portion of the above-mentioned empty space.

[0086] The cover member may be provided on the vacuum cleaner body or the cleaning unit. In the present embodiment, the cover member is formed protrudingly on each side of the rear end of the cleaning unit, and is arranged to cover the outer surface of the vacuum cleaner body.

[0087] The cover member is positioned to fill at least a portion of the empty space, i.e., the empty space between the vacuum cleaner body and the cleaning unit. Accordingly, an obstacle can be prevented from entering the empty space, or a structure can be implemented that allows for easy removal of an obstacle even if it is trapped in the empty space.

[0088] The cover member formed protruding from the cleaning unit can be supported on the outer surface of the cleaner body.

[0089] If the cover member protrudes from the main body of the vacuum cleaner, the cover member may be supported by the rear of the cleaning unit. According to the above structure, when the cleaning unit collides with an obstacle and receives an impact, a portion of the impact is transmitted to the main body of the vacuum cleaner, thereby dispersing the impact.

[0090] The cleaning unit can be detachably connected to the vacuum cleaner body. When the cleaning unit is separated from the vacuum cleaner body, a mop module (not shown) can be detachably connected to the vacuum cleaner body to replace the separated cleaning unit.

[0091] Therefore, if the user wants to remove dust from the floor, he / she can attach a cleaning unit to the vacuum cleaner body, and if he / she wants to clean the floor, he / she can attach a mop module to the vacuum cleaner body.

[0092] When the cleaning unit is mounted on the main body of the vacuum cleaner, the mounting can be guided by the cover member described above. That is, by positioning the cover member to cover the outer surface of the main body of the vacuum cleaner, the relative position of the cleaning unit with respect to the main body of the vacuum cleaner can be determined.

[0093] The cleaning unit may be equipped with casters. The casters are configured to assist the movement of the robot (10) and also to support the robot (10).

[0094]

[0095] Meanwhile, a robot according to one embodiment of the present invention can search for a sound source and perform cleaning for a predetermined range based on the searched sound source.

[0096] In the case of the house, the space is narrow due to various furniture and objects, so the sound is easily reflected by these objects or walls, and is acquired as noise data. Therefore, when the robot follows the direction of the angle with only the sound angle of the sound sensor, the wrong location is often detected.

[0097] Especially within hallways connecting rooms, sound reflections from the walls on either side often point toward the walls, or are sometimes captured irregularly, often resulting in failures in sound localization. To detect specific sounds within a home and extract their locations, a process for identifying and eliminating spurious noise data is crucial.

[0098] The difficulties in detecting the location of sounds in the home are as follows:

[0099] The first reason is the precision of the sound sensor.

[0100] Triangulation techniques are often used to detect the location of a sound source. By measuring the angle of the sound at two points and knowing the robot's position at those two points, the location of the sound source can be determined using the equation of two straight lines. For sound sensors, the direction of the sound is detected relative to the robot, but the distance is unknown. The angular accuracy of sound sensors is often inaccurate, typically within 5-10 degrees. Therefore, when triangulation is used, the farther the sound source is, the more inaccurate the location will be due to angular error. For the robot, however, the closer the sound source is, the more accurate the location. While increasing the number of microphones can improve this accuracy, the higher the number of microphones and the higher the CPU requirements, which leads to higher material costs.

[0101] The second reason is that most sounds in the home are acquired as noise data.

[0102] When detecting sounds such as a hair dryer, it is difficult to detect because noise such as TV or music sounds in the house acts as noise, and even when there is no TV or music sound, when detecting the sound of a hair dryer reflected from walls or objects in the house, the direction of the sound continuously changes, making it difficult to detect the exact location of the hair dryer.

[0103] The present invention can solve these problems and provide a method for finding a more accurate location of a sound source.

[0104] The robot (10) of the present invention detects the gate position and the Bornoi node using an existing map, and when the user sets the location (zone area) (or area of ​​interest) where sound is used, the robot uses these three pieces of location information to conduct sound search.

[0105] The robot (10) of the present invention moves along a path that allows for the shortest distance by using Voronoi nodes and a zone area to find a sound search within a short period of time. If a sound is detected in the zone area during the search, the robot ends the sound search, and if no sound is detected in the zone area, the robot continues to detect the sound.

[0106] In addition, the robot (10) of the present invention determines that the sound is not present in the room when the direction of the sound is directed outside the room at the gate position, thereby stopping the room search and moving to the next room. This search technique can speed up the sound search while reducing the error rate.

[0107] The robot (10) of the present invention determines the location of sound where sound is concentrated as the location of sound through dbscan clustering based on the location of sound obtained through triangulation during search in order to reduce noise reflected on a wall or noise generated in the user's daily life, and also determines whether the location of sound has been reached by obtaining an envelope curve of raw sound data and using the minimum + maximum value (min + max value), and can output the location of sound that is robust to noise.

[0108] Figure 2 is a conceptual diagram to explain why it is difficult to determine the source of a sound when using a conventional method.

[0109] In the past, to find the source of a sound, a sound sensor in the form of a microphone array on a robot was used to detect the location of the sound.

[0110] For example, in the past, a robot moved by finding the location of a sound by using the shifted phase difference between the original sound and the reflected sound reflected from a reflective surface using the Doppler effect.

[0111] Conventional robots predicted the location of the original sound by looking at the size of the energy (loudness of the sound) among the reflected sounds detected by the original sound.

[0112] However, when conducting actual tests, there is a problem in that when the robot is close to a wall, the reflected sound reflected from the wall has more energy than the original sound, and in such cases, there is a high possibility of errors occurring when searching for the location of the sound.

[0113] Referring to Figure 2, the sound direction data obtained from the sensor varies depending on the position of the robot.

[0114] In cases where the sound is far away, such as in (a), the sound will randomly head in the direction from which it is coming, and if the distance is far and the location is determined by triangulation, inaccurate results will be obtained.

[0115] In cases such as (b) and (c), where there is a wall or a highly reflective object nearby, the direction of the reflected sound is detected, and the sound is sensed as coming from the wall or object. Therefore, in cases (a), (b), and (c), using the detected sound data to detect a location may yield inaccurate results.

[0116] (d) To measure the exact angle of the sound, you must go near the sound and receive the direct sound, and only by using this data can you detect the exact location of the sound source.

[0117] For this reason, by attaching a sensor to the robot to continuously track the direction of sound, when the robot moves, the direction of the sound is inaccurate if the sound is far away, and the direction of the sound continuously changes depending on the location due to diffuse reflection by walls or nearby objects, making it impossible to track the sound as it moves to the reflected wall or object.

[0118] The present invention can solve these problems and provide a method for detecting the location of a sound source more accurately.

[0119] Figure 3 is a flowchart for explaining a representative sound source search method of the present invention.

[0120] The robot of the present invention may include a driving unit (130) formed to move the main body of the robot, a microphone (122) (or sensing unit) for receiving external sounds, and a control unit (180) for controlling them.

[0121] The control unit (180) can obtain a Bornoi graph based on a map of the space in which the robot moves, and extract Bornoi nodes from the Bornoi graph (S310).

[0122] Specifically, the control unit (180) can obtain a map (or a map for the space in which the robot exists) corresponding to the space in which the robot exists (or a space configured for the robot to move (e.g., a house, a space set to be cleaned, etc.)).

[0123] The process of acquiring a map corresponding to the space where the robot exists can be applied to a technology / process for generating a general map, such as generating a map while the robot actually drives, or generating a map using wall information sensed through a sensing unit.

[0124] The control unit (180) can extract rooms and rooms using a map corresponding to the space where the robot exists (S320), and extract Bornoi nodes for each room from the map.

[0125] At this time, the control unit (180) can obtain a Bornoi graph using the map, and determine (extract) nodes that satisfy preset conditions in the Bornoi graph (points from which paths (branches) formed for the robot to drive extend) as Bornoi nodes. The process of determining Bornoi nodes will be described in more detail later.

[0126] The control unit (180) can control the robot to search for the sound source based on the area of ​​interest set by the user and the Bornoi node set for each room (S330).

[0127] Specifically, the control unit (180) can determine the order in which the robot will pass so as to satisfy preset conditions by using the Point Of Interest (or Zone area) set by the user and the Bornoi nodes extracted for each room.

[0128] The control unit (180) can control the driving unit (130) so that the robot passes through at least one of the region of interest and the Bornoi node in the above order to find the source of the external sound.

[0129] In addition, when the control unit (180) determines that the direction of an external sound is outside the room corresponding to the visit when the robot passes through the door, the control unit can control the driving unit to bypass at least one of the regions of interest and Bornoi nodes existing in the room corresponding to the visit.

[0130] Specifically, the control unit (180) can determine whether the source of the sound is outside the room while moving toward the region of interest or the Bornoi node located in each room. If the source of the sound is outside the room, the control unit (180) can control the robot (drive unit) to move to another region of interest or the next Bornoi node according to a determined order without passing through the region of interest or the Bornoi node located in the room (inside the room) of the visit (S340).

[0131] Figures 4, 5, 6, 7, 8, 9, 10 and 11 are conceptual diagrams for explaining the sound source search method examined in Figure 3.

[0132] The control unit (180) of the robot (10) can obtain a Bornoi graph indicating a path that the robot can drive on a map using the Bornoi algorithm.

[0133] For example, referring to FIG. 4, the control unit (180) of the robot (10) can obtain a Bornoi graph (410), as shown in FIG. 4(b), based on a map (400) of the space in which the robot moves, as shown in FIG. 4(a).

[0134] Specifically, the control unit (180) can obtain a new map (405) in which values ​​are expressed as larger values ​​the farther away from the wall by performing distance transformation on the map, as shown in FIG. 4(b), and can obtain a Bornoi graph (410) by tracking changes in pixel values ​​in the new map (405) in a direction within a preset range.

[0135] The control unit (180) can set the part where the graph (or path, branch) intersects in the Bornoi graph (410) as a Bornoi node (420), as shown in FIG. 4(c), and can set the Bornoi node so that at least one Bornoi node is included in each room.

[0136] Referring to Figure 4, the following describes the process of calculating a Bornoi graph before extracting Bornoi nodes.

[0137] The control unit (180) can obtain a new map (405) in which, by performing distance transformation on a two-dimensional map, values ​​are expressed as larger values ​​as distance from the wall increases. Here, the value of each pixel is proportional to the distance to the nearest wall. In Fig. 4(b), large values ​​are expressed in white.

[0138] The control unit (180) can obtain the Bornoi graph (410) shown in Fig. 4(b) by starting from the largest value in the distance-transformed map and tracking in the direction of the smallest change in value.

[0139] The control unit (180) can use the part where graphs intersect in the Bornoi graph as a Bornoi node and can select (determine) a Bornoi node (420) according to a preset method so that at least one node exists in each room by utilizing the room detection result.

[0140] Since the Bornoi graph is drawn from the largest empty space to the smallest empty space, two or more Bornoi nodes (or intersections) can appear in a complex shape other than a square, such as a 'ㄷ' shape.

[0141] Figure 5 is a drawing showing the process of extracting rooms and doors using a map in the robot (10) of the present invention.

[0142] The control unit (180) can divide the map into blank space (white), wall (black), and unobserved (gray), as shown in Fig. 5(a).

[0143] The control unit (180) checks whether the empty space is separated while gradually increasing (expanding) the size of the wall.

[0144] The control unit (180) can store the state when the empty space is separated to the extent that information of each room can be expressed, as shown in Fig. 5(b).

[0145] The control unit (180) can expand the size of the room area by targeting the empty space in the stored room information, as shown in Fig. 5(c). The control unit (180) stores the part where each room area meets as a visit (510).

[0146] In addition, the control unit (180) may determine a room and a visit by determining a part of the map where the distance between walls is within a certain distance, or by using an artificial intelligence algorithm that outputs a visit as an output value on the map through deep learning.

[0147] The control unit (180) determines the nodes to be visited using the area of ​​interest, Bornoi node (420), and room and visit (510) information, and determines the strategy to be taken during the visit process to search for the sound source.

[0148] Figure 6 is an explanation for obtaining data necessary before the robot of the present invention searches for the source of a sound.

[0149] The robot's control unit (180) can detect Bornoi nodes using a grid map and obtain the number of nodes by optimizing them.

[0150] The control unit (180) can detect Bornoi nodes according to preset conditions. For example, the control unit (180) can detect one or more Bornoi nodes per room, calculate the complexity of the room, and detect multiple Bornoi nodes if the complexity of the room exceeds a threshold value.

[0151] For example, the control unit (180) acquires one node for each room, and in complex cases such as a bed or appliance, detects multiple nodes (for example, up to two). In addition, the control unit (180) detects at least two to three nodes for a living room, etc., based on the size of the room.

[0152] These nodes are representative locations (or points) that the robot will visit when searching for the source of a sound. A large number of nodes means more nodes to visit, which increases the time it takes to search for the source. A small number of nodes speeds up the search time, but may result in inaccurate sound location.

[0153] Additionally, the control unit (180) detects the gate position data of each room to perform a quick search. This data includes gate position information in grid units.

[0154] The control unit (180) can use gate position data to determine whether there is sound in a room during sound search, and if there is no sound, can be used to immediately exit the room without entering. This can save time during sound search.

[0155] That is, the control unit (180) can control the driving unit (130) to determine that the sound is in the room if the direction of the sound is toward the room from the location of the gate, and to determine that the sound is not in the room if the direction of the sound is toward the outside of the room, and to exit the room without searching the room.

[0156] Because sound bounces off walls indoors, it's difficult to accurately estimate its location without exploring the entire house. To complete the search quickly, we must use the given map to identify Bornoi nodes and select those with high visitability.

[0157] First, let's examine two important conditions: 1) each room must have at least one node. 2) large or angular rooms may have two or more nodes, including blind spots. To satisfy these two conditions, the control unit (180) extracts nodes in the following order.

[0158] The control unit (180) can extract nodes by extracting only nodes with three or more graphs (links (branches)) from the Bornoi graph, calculating a distance map of the entire map, clustering nearby nodes, and then using the distance map to select the node farthest from the wall (the widest area of ​​the room).

[0159] The control unit (180) may apply additional methods, such as adjusting the degree of proximity to be included in a cluster according to the number of nodes in the cluster, since the criteria for nearby nodes may vary depending on the size of the room.

[0160] As shown in FIG. 6, the control unit (180) can obtain a Bornoi node using the method mentioned above, and can determine the order of visited locations for quick search using the Bornoi node obtained in this way and the information on the area of ​​interest set by the user (or the Zone area registered by the user).

[0161] First, the control unit (180) can trigger the robot with a specific sound (such as a hairdryer sound). When the robot is triggered with a specific sound, the control unit (180) can detach the robot's main body from the docking station and then acquire angular data of external sounds received through the microphone.

[0162] The control unit (180) can determine the direction of the sound using an algorithm that determines the direction from which the sound received through the microphone (or sound sensor) is coming, and based on this, when an external sound is received, data on the angle at which the external sound is received can be obtained.

[0163] The control unit (180) uses this value to select the location of the zone of interest (Zone area) having the closest angle and distance to the angle (sound angle) determined according to the direction in which external sound is received from the initial starting position as the first search location.

[0164] That is, the control unit (180) can determine whether a region of interest set by the user exists, and if a region of interest exists, determine the order to first pass through the region of interest closest to the robot.

[0165] Afterwards, the control unit (180) uses the global planner as the next search location to select the location of the zone area with the shortest path.

[0166] In this way, the control unit (180) can determine the search order so that, once the search locations of all zone areas are selected, the node of Bornoi with the shortest path is sequentially selected.

[0167] That is, when multiple regions of interest are set, the control unit (180) can determine the order to pass through the multiple regions of interest at the shortest distance and then pass through the Bornoi node.

[0168] Meanwhile, if the zone area and the Bornoi node are within a certain distance, the control unit (180) can combine the locations of the Bornoi node and the zone area to determine the order in which only one of the two is visited for faster search. This method enables faster search.

[0169] This series of algorithms allows the robot to quickly perform sound exploration.

[0170] Meanwhile, the control unit (180) may determine the order so that, instead of searching the zone area and the Bornoi node separately as mentioned above, they are combined and the closest zone area or Bornoi node from the initial position is set as the first search position, and then the closest zone area or Bornoi node along the path is sequentially searched.

[0171] That is, the control unit (180) may determine the order to go through the closest area of ​​interest (or the closest Bornoi node from where the robot is located) and then go through the closest area of ​​interest or Bornoi node from the closest area of ​​interest.

[0172] Meanwhile, the control unit (180) may determine whether the region of interest set by the user exists, and if the region of interest does not exist, may determine the order to first pass through the Bornoi node closest to the robot.

[0173] Thereafter, the control unit (180) can determine the order to go through the nearest Bornoi node and then go through the nearest Bornoi node from the nearest Bornoi node.

[0174] Since the efficient algorithm among the two methods above will differ depending on the location of the user-set zone area and the environment of the space in which the robot exists, initially, the robot can perform sound detection in these two modes randomly to obtain search time statistics, and after a certain number of times (e.g., 4-5 times), the search selection method can be selected based on the fastest search method. This can result in a pattern similar to a kind of reinforcement learning in which the robot operates randomly and pursues the efficient result among them.

[0175] Figure 7 shows how a robot finds a location (zone area) where a user will use sound (e.g., a hair dryer) and how the robot finds the source of the sound when the zone area is located there.

[0176] The control unit (180) recognizes the sound of a hair dryer with a deep learning engine and detects the location of the sound by searching for the node location closest to the angle of the sound detected by the microphone (sound sensor) from the initial starting position.

[0177] For example, if a user registers an area (zone area) where he or she mainly uses a hair dryer (e.g., a location where a dressing table or full-length mirror is located in a room), the control unit (180) stores data about this, and when searching for the sound source, it first starts searching from the initial location to the area of ​​interest (zone area) closest to the angle of the sound generation.

[0178] For example, as shown in FIG. 7, when zones of interest 1 to 3 (Zone area1 to 3) are registered, the control unit (180) first searches for zone area1 that is closest to the initial direction (a) of the sound from the initial position, and in this case, when a sound is detected within a certain distance of a zone area that the user has previously designated, the robot ends the sound search, stores the location of the sound, ends the sound search, and returns to the docking station to dock with the cleaning module for cleaning.

[0179] At this time, the control unit (180) can dock with the vacuum cleaner module and move to the location of the stored sound and perform cleaning for a predetermined area size after a certain period of time has passed or when it is determined that the sound has stopped (when the user has finished using the hair dryer and the sound has stopped).

[0180] Figure 8 is a diagram illustrating a method for searching for a sound source using gate position information for quick search.

[0181] The control unit (180) can determine the direction of an external sound received through a microphone, and when the robot arrives at the location of the visit, determine whether the direction from which the external sound is heard is inside or outside the room corresponding to the visit.

[0182] Specifically, the control unit (180) can determine the direction of external sound received through the microphone (sound direction, angle from which the sound is heard).

[0183] Thereafter, when the robot arrives at the location of the visit while performing a search via the nodes in the order described above, the control unit (180) can determine whether the direction from which the external sound is heard is inside or outside the room corresponding to the visit.

[0184] The control unit (180) can predict the presence or absence of sound in the room by using the gate position extracted from FIG. 5 and knowing the direction of the sound from the gate position.

[0185] In a case like Figure 8(a), there is sound in the room, and the direction of the sound from the visiting location is directed towards the room. In this case, the control unit (180) searches according to the order of search locations based on the previously determined order of passage.

[0186] In cases like Figure 8(b), when there is no sound in the room, the direction of the sound is directed outside the room. In this case, even if there is a Bornoi node or area of ​​interest (zone area) to be searched in the room, the control unit (180) can skip this search location for a quick search and immediately exit the room and move to the next room (next node).

[0187] That is, the control unit (180) can control the driving unit so that the robot does not pass through a region of interest or a Bornoi node existing in the room corresponding to the visit when the direction from which the external sound is heard is outside the room corresponding to the visit.

[0188] At this time, the control unit (180) determines that the external sound is received from outside the room corresponding to the visit, and can process it as having passed through the area of ​​interest or the Bornoi node existing in the room, even if it does not pass through the area of ​​interest or the Bornoi node existing in the room corresponding to the visit.

[0189] In order to reduce search time, the present invention utilizes gate position data during sound search. When the robot is located at the gate position, the sound angle received from the microphone (or sound sensor) is measured to determine whether it is directed toward the room or outward. If it is directed toward the outside of the room, the robot determines that there is no sound source in the room, stops searching for nodes in the room, and exits the room. Through this, the present invention can save time in finding the location of the sound source.

[0190] Gate position information can be obtained by inflating obstacle information using map information or by obtaining deep learning learning information using image information.

[0191] Figure 9 is a diagram for explaining a case where the sound source is not in the zone area.

[0192] The control unit (180) can move from the initial starting position to the area of ​​interest 1 (zone area1) close to the initial direction of the sound (a) based on the algorithm for determining the order of passage described above, and sequentially search and move in the order of area of ​​interest 2 (zone area2) -> room 2 Bornoi node -> area of ​​interest 3 (zone area3) -> room 3 Bornoi node.

[0193] The control unit (180) searches room 1 because the sound is directed toward the room at gate position 1 (Gate pos1), and exits room 2 and room 3 without entering them because the sound is directed outward at gate positions 2 and 3 (gate pos2, 3), or enters room 2 and room 3 briefly for a turn and then exits without performing a sound search.

[0194] The control unit (180) searches all rooms (even rooms where the sound direction is directed outward from the room and thus no search is performed are processed as searched rooms) because the sound source is not detected in the zone area, and finally stores the location of the detected sound as the final result and then arrives at the docking station to complete the sound search.

[0195] When the microphone (sound sensor) described above receives an external sound, the control unit (180) (or sound sensor) can calculate the direction of the received sound (initial direction of the sound, direction of the sound, angle of the sound).

[0196] Meanwhile, the control unit (180) provided in the robot (10) of the present invention can receive input from multiple directional microphones and output (judge) the horizontal angle and vertical angle (elevation) of the sound.

[0197] Depending on the robot's configuration, the sound sensor's microphones can be positioned horizontally or in both horizontal and vertical planes. This arrangement can be determined depending on the robot's configuration.

[0198] When the sensor's microphones are placed only in the horizontal plane, only the sound direction is output. However, when a combination of horizontally placed microphone sensors are attached in multiple layers in the vertical plane, both the sound direction and the elevation angle are output. Here, the sound direction data is used to triangulate the sound's location, and the elevation angle is used to determine whether the sensor (robot) is near the sound.

[0199] When using the elevation angle, it is possible to determine whether the location of the sound source obtained in real time through the search is the final location, so the search can be terminated by determining the final location of the source in the middle of the search. However, the sensor price is expensive because it requires a large number of microphones and a high-performance CPU.

[0200] The control unit (180) uses sound dB (sound volume) when the microphone is positioned only in a horizontal plane and elevation angle data is not output. In this case, the control unit (180) can determine the point where the sound volume is maximum as the final location of the sound.

[0201] However, since it is difficult to determine the point where the maximum sound is generated as the final sound source location during the search, the control unit (180) can determine the final location during the search only when the location of the sound where the maximum sound is generated for a certain period of time (approximately 5 seconds) matches the location of the zone area designated by the user.

[0202] The control unit (180) can determine the final sound result after searching all Bornoi nodes if the sound location is not detected in the zone area.

[0203] In the case of a hair dryer, the absolute size of the sound cannot be used because the intensity of use and the noise characteristics of the hair dryer itself are different, and the relative size is used. Therefore, the control unit (180) determines the final sound location after searching the entire zone area and the Bonnoy node.

[0204] The control unit (180) can control the driving unit to move the robot to the initial position it moved to when the route to the area of ​​interest set on the map and the Bornoi node is all completed.

[0205] When the search for the sound source for the area of ​​interest and the Bornoi node is completed and the sound source is determined, the control unit (180) can control the driving unit (130) to move to the initial position (docking station), combine the cleaning unit at the initial position, and then move to the determined sound source to perform cleaning.

[0206] The present invention can control the robot (10) to separate the cleaning unit when searching for a sound source, and to combine the cleaning unit when performing cleaning after the sound source search is complete. Through this, the present invention can reduce the battery consumption required for the robot's movement when searching for a sound source, and can accelerate the movement speed, enabling rapid sound source search.

[0207] Meanwhile, the control unit (180) can perform a sound location estimation algorithm to determine the source of external sound while moving the robot.

[0208] Figure 10 is a diagram for explaining a sound location estimation algorithm.

[0209] The control unit (180) moves along a given search path and stores N pieces of sound data at a certain distance. At this time, the control unit (180) measures the average dB (decibels) of N pieces of sound data and updates N pieces of sound data when the maximum of this value is updated.

[0210] At this time, the dB size of the sound is an LPF (low pass filtering) signal, so short-range noisy sounds are filtered out.

[0211] The sound data of N obtained in this way may include the robot's position, sound angle, sound elevation, sound raw data, and sound decibel value.

[0212] The control unit (180) uses the 1st to Nth data of the sound data to obtain the equation of the straight line y= ax+b at each point and obtains the intersection point using the equation of the straight line for two different robot positions. The number of intersection points obtained in this way is N*(N-1), and the control unit (180) can estimate (assume) that the sound source is within this number.

[0213] When a robot searches for sound, the closer it gets to the location of the sound, the larger the dB size of the sound data becomes. Ultimately, the location of the sound source is determined when the dB is the largest, so the control unit (180) can obtain the final location of the sound by measuring the location of the sound using triangulation and updating the time when the dB of the sound is the largest.

[0214] Alternatively, the elevation value of the sensor output data can be used. The elevation value increases as the sensor approaches the sound, and by using this, the control unit (180) can determine whether the robot has reached the vicinity of the sound source.

[0215] The robot of the present invention can determine the final location of the sound source by using both dB size and elevation, since the type of sensor changes depending on the shape of the robot.

[0216] Using elevation values ​​can yield more reliable results, but the sensor that outputs elevation values ​​must be equipped with more microphones, which can increase material costs.

[0217] Figure 11 is a conceptual diagram explaining the process of acquiring sound data while the robot moves and using the data to determine the location of the sound.

[0218] The control unit (180) of the robot (10) can receive sound raw data and sound direction data from a microphone (sound sensor).

[0219] The control unit (180) can filter sound raw data to remove short noise sounds and obtain an envelope curve.

[0220] The control unit (180) can obtain the maximum (max) and minimum (min) of the envelope curve and obtain the maximum by adding the values.

[0221] The control unit (180) can determine the filtered sound level as the sum of the maximum and minimum of the envelope curve. Since the maximum value of this filtered sound level increases as it gets closer to the sound, the control unit (180) can acquire N sound data (or sound raw data) every time the maximum value of the filtered sound level is updated and update the location result of the sound source.

[0222] In this way, the control unit (180) can finally obtain the location of the sound source by continuously updating the estimated location (Sound Pos. (Position)) of the sound source whenever the filtered sound level becomes maximum.

[0223] When the elevation angle of the sound sensor cannot be used, the control unit (180) calculates the sound level decibel value and updates the sound data when N sound level values ​​become the maximum. The N data are data that are continuous in time.

[0224] The control unit (180) uses data from two different points of the acquired N data to obtain the equation of a straight line and obtain the intersection point thereof.

[0225] The control unit (180) clusters (clusters (DBSCAN clustering)) points that are within a certain distance from the gathered points and have a certain density (a certain number of points or more), and determines the point with the largest number of points among the clustered group as the location of the sound source.

[0226] FIG. 12 and FIG. 13 are flowcharts for explaining a method for searching for a sound source and a method for estimating a sound location of a robot according to one embodiment of the present invention.

[0227] Referring to FIG. 12, the control unit (180) extracts a Bornoi node and a visit location (gate pose) using map data and retrieves data of a zone area set by the user (S1202).

[0228] The control unit (180) obtains the sound direction (A) from the sensor data at the initial starting position (S1204), and if there is zone area data set by the user (S1206), selects the zone area closest to the direction A (S1208), and if not, selects the closest Bornoi node (S1210) to set as the starting point, and plans / calculates the shortest distance to visit all remaining nodes to calculate the shortest path (S1212).

[0229] The control unit (180) moves along the planned path while starting the sound location estimation algorithm (S1214, S1216, S1218).

[0230] When passing through the gate pose, the control unit (180) determines whether the direction of the sound is inside or outside the room (S1220). If the direction of the sound is outside the room, the control unit skips the node and searches for the next node. If the direction of the sound is inside the room, the control unit searches for nodes inside the room and moves to the next node (S1218).

[0231] The control unit (180) determines whether the direction of the sound is inside the room or outside when passing through the gate pose (S1220), and if the direction of the sound is inside the room, searches for the location of the sound while passing through the region of interest or Bornoi node existing in the room (S1224).

[0232] The control unit (180) terminates the search if there is an estimated sound location in the zone area (or within a certain distance from the zone area) during the search (S1226, S1228). Otherwise, after searching all nodes (S1222), the final location of the estimated sound is saved as the final result and the search is terminated (S1228).

[0233] Referring to Fig. 13, when the sound location estimation algorithm starts (S1216), the control unit (180) acquires sound direction data (sound angle data) while the robot moves (S1302) and then obtains an envelope curve of sound raw data (S1304).

[0234] The control unit (180) can acquire N pieces of sound raw data (S1308) when the filtered sound data representing the maximum plus minimum value (max+min value) of the envelope curve is the maximum (S1306), and output (determine) the sound source location (Sound POS) by triangulating and clustering (S1310, S1312).

[0235] The effects of the acoustic recognition device and the control method thereof according to the present invention are described as follows.

[0236] According to at least one of the embodiments of the present invention, the present invention can search all rooms using a region of interest to reduce search errors and increase the success rate of finding a sound source.

[0237] The present invention can significantly reduce search time by using the visit information of each room to move to another room without searching the room if the direction of the sound is outside the room.

[0238] The present invention can enhance noise vulnerability by obtaining an envelope curve of raw sound data in the process of detecting a sound location and determining the sound source using the maximum and minimum values ​​of the envelope curve.

[0239] The present invention described above can be implemented as a computer-readable code on a medium in which a program is recorded. The computer-readable medium includes all types of recording devices that store data that can be read by a computer system. Examples of the computer-readable medium include a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc., and also includes media implemented in the form of a carrier wave (e.g., transmission via the Internet). In addition, the computer may include a control unit (180) of a robot (10) according to an embodiment of the present invention.

[0240] Accordingly, the above detailed description should not be construed as limiting in all respects, but rather as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are intended to be included within the scope of the present invention.

Claims

1. A driving unit formed to move the main body of the robot; a microphone for receiving external sounds; and Extract rooms and doors using a map corresponding to the space where the above robot exists, Extract Bornoi nodes by room from the above map, Using the Bornoi nodes extracted by the user-defined area of ​​interest and room, the order in which the robot will pass is determined so as to satisfy the preset conditions. Including a control unit that controls the driving unit so that the robot passes through at least one of the region of interest and the Bornoi node in the above order to find the source of the external sound, The above control unit, A robot characterized in that when the robot passes through the door, if the direction of the external sound is determined to be outside the room corresponding to the door, at least one of the region of interest and Bornoi node existing in the room corresponding to the door is not passed through.

2. In paragraph 1, The above control unit, A robot characterized in that it obtains a Bornoi graph based on a map of a space in which the robot moves, and extracts Bornoi nodes that satisfy preset conditions from the Bornoi graph.

3. In paragraph 2, The above control unit, A robot characterized in that it obtains a new map in which a larger value is expressed as the distance from the wall is increased by performing a distance transformation on the above map, and obtains a Bornoi graph by tracing the change in pixel value in the new map in a direction within a preset range.

4. In paragraph 3, The above control unit, In the above Bornoi graph, the part where the graphs intersect is set as a Bornoi node. A robot characterized in that the Bornoi nodes are set so that each room includes at least one Bornoi node.

5. In paragraph 1, The above control unit, A robot characterized in that it determines whether a region of interest set by the user exists, and if a region of interest exists, it determines the order to first pass through the region of interest closest to the robot.

6. In paragraph 5, The above control unit, A robot characterized in that, when multiple regions of interest are set, the order is determined so as to pass through multiple regions of interest by the shortest distance and then through a Bornoi node.

7. In paragraph 5, The above control unit, A robot characterized in that the order is determined so as to pass through the closest region of interest and then pass through the closest region of interest or Bornoi node in the closest region of interest.

8. In paragraph 1, The above control unit, A robot characterized in that it determines whether a region of interest set by the user exists, and if the region of interest does not exist, it determines the order to first pass through the Bornoi node closest to the robot.

9. In paragraph 8, The above control unit, A robot characterized in that the order is determined to go through the nearest Bornoi node and then go through the nearest Bornoi node from the nearest Bornoi node.

10. In paragraph 1, The above control unit, A robot characterized in that it determines the direction of an external sound received through the microphone, and when the robot arrives at the location of the visit, it determines whether the direction from which the external sound is heard is inside or outside the room corresponding to the visit.

11. In Article 10, The above control unit, A robot characterized in that, when the direction from which the external sound is heard is outside the room corresponding to the visit, the driving unit is controlled so that the robot does not pass through a region of interest or a Bornoi node existing in the room corresponding to the visit.

12. In paragraph 11, The above control unit, A robot characterized in that it determines that the external sound is received from outside the room corresponding to the visit and processes it as having passed through the region of interest or Bornoi node existing in the room corresponding to the visit even if it does not pass through the region of interest or Bornoi node existing in the room corresponding to the visit.

13. In paragraph 1, The above control unit, A robot characterized by performing a sound location estimation algorithm to determine the source of the external sound while moving the robot.

14. In paragraph 13, The above control unit, A robot characterized in that when the above-described region of interest and the above-described Bornoi node set on the above-described map are both completed, the driving unit is controlled to move to the initial position to which the robot moved.

Images