Robot and controlling method thereof

Abstract

A robot is disclosed. The robot includes a drive unit, a camera, a plurality of microphones arranged in different directions, and a processor that, when an audio signal input through the plurality of microphones is identified as an audio signal corresponding to a cough sound, identifies the direction of occurrence of the audio signal and controls the camera to capture the identified direction; when a user not wearing a mask is identified in the image acquired through the camera, identifies a sterilization area based on the user's location and controls the drive unit to move to the identified sterilization area.

Description

Technology Field

[0001] The present invention relates to a robot that performs a sterilization function and a method for controlling the same. Background Technology

[0002] With the recent emergence of the highly contagious coronavirus, preventing transmission has become a pressing social issue. While viruses are generally transmitted through droplets from coughing or contact, conventional sterilization robots face the problem of being unable to perform automated sterilization functions because they sterilize droplets or contact points through user operation. The problem to be solved

[0003] The present disclosure is in accordance with the aforementioned necessity, and the purpose of the present disclosure is to provide a robot capable of identifying a sterilization area on its own without user operation and a method for controlling the same. means of solving the problem

[0004] A robot according to one embodiment of the present invention for achieving the above objectives may include a driving unit, a camera, a plurality of microphones arranged in different directions, and a processor that, when an audio signal input through the plurality of microphones is identified as an audio signal corresponding to a cough sound, identifies the direction of occurrence of the audio signal and controls the camera to photograph the identified direction, and when a user not wearing a mask is identified in an image acquired through the camera, identifies a sterilization area based on the location of the user and controls the driving unit to move to the identified sterilization area.

[0005] Here, the processor can identify a critical range area based on the user's location and the strength of the audio signal as the sterilization area.

[0006] In addition, the processor can determine the size of the sterilization area based on the number of users not wearing the mask.

[0007] In addition, when multiple users who are not wearing the mask are identified, the processor can identify the sterilization area based on the location of each of the multiple users.

[0008] In addition, the processor may determine at least one of the size of the sterilization area or the sterilization intensity based on whether the location identified by the user belongs to a preset area.

[0009] Here, the aforementioned preset area may include at least one of an area where an object with a high frequency of contact by users is located, an area with a high frequency of visits by users, or an area with a low frequency of mask wearing by users.

[0010] Meanwhile, the processor identifies whether the input audio signal is an audio signal corresponding to the cough sound using a first neural network model, and the first neural network model may be a model trained to identify whether the input audio signal includes a cough sound.

[0011] Additionally, the processor identifies whether the acquired image includes a user who is not wearing a mask using a second neural network model, and the second neural network model may be a model trained to identify whether the input image includes a user who is not wearing a mask.

[0012] Additionally, the processor further includes a distance sensor, and when a plurality of sterilization areas are identified, the processor identifies the direction and distance of each of the plurality of sterilization areas using the distance sensor, and can set a driving path for sterilization work based on the identified direction and distance.

[0013] In addition, it further includes a sterilization device that performs a sterilization function, and when the robot moves to the identified sterilization area, the sterilization device can be controlled to perform a sterilization function.

[0014] Meanwhile, a control method for a robot comprising a plurality of microphones and a camera arranged in different directions according to an embodiment of the present invention may include the steps of: identifying the direction of generation of the audio signal based on the arrangement direction of each of the plurality of microphones when the audio signal input through the plurality of microphones is identified as an audio signal corresponding to a cough sound; capturing the identified direction through the camera, and when a user not wearing a mask is identified in the image obtained through the camera, identifying a sterilization area based on the location of the user; and moving to the identified sterilization area.

[0015] Here, the step of identifying the sterilization area may identify a critical range area as a sterilization area based on the user's location and the strength of the audio signal.

[0016] In addition, the step of identifying the sterilization area may determine the size of the sterilization area based on the number of users not wearing the mask.

[0017] In addition, the step of identifying the sterilization area may identify the sterilization area based on the location of each of the multiple users when multiple users who are not wearing the mask are identified.

[0018] In addition, the step of identifying the sterilization area may determine at least one of the size of the sterilization area or the sterilization intensity based on whether the user's identified location belongs to a preset area.

[0019] Here, the aforementioned preset area may include at least one of an area where an object with a high frequency of contact by users is located, an area with a high frequency of visits by users, or an area with a low frequency of mask wearing by users.

[0020] Meanwhile, the step of identifying the direction in which the above audio signal is generated is to identify whether the input audio signal is an audio signal corresponding to the cough sound using a first neural network model, and the first neural network model may be a model trained to identify whether the input audio signal includes a cough sound.

[0021] Additionally, the step of identifying a sterilization area based on the location of the user may identify whether the acquired image includes a user not wearing a mask using a second neural network model, and the second neural network model may be a model trained to identify whether the input image includes a user not wearing a mask.

[0022] In addition, when a plurality of sterilization areas are identified, the method may further include the step of identifying the direction and distance of each of the plurality of sterilization areas, and the step of setting a driving path for sterilization work based on the identified direction and distance.

[0023] In addition, when the robot moves to the identified sterilization area, it may further include a step of performing a sterilization function. Effects of the invention

[0024] According to various embodiments of the present disclosure, an area requiring sterilization can be sterilized using only a robot without the need to use other sensors installed in an indoor space. Brief explanation of the drawing

[0025] FIG. 1 is a drawing for explaining a method of operating a robot according to one embodiment of the present disclosure. FIG. 2 is a block diagram for explaining the configuration of a robot according to one embodiment of the present disclosure. FIGS. 3a and 3b illustrate the size of a sterilization area corresponding to the loudness of a cough sound according to one embodiment of the present disclosure. FIGS. 4a and 4b illustrate the size of a sterilization area corresponding to the number of people not wearing a mask according to one embodiment of the present disclosure. FIGS. 5A and 5B are drawings for explaining a method for determining a sterilization area for sterilizing a plurality of sterilization points according to one embodiment of the present disclosure. FIG. 6 illustrates the size and sterilization intensity of a sterilization area corresponding to a location containing a sterilization point according to one embodiment of the present disclosure. FIG. 7a is a diagram illustrating an audio signal processing method using a neural network model according to one embodiment of the present disclosure. FIG. 7b is a drawing for explaining an image processing method using a neural network model according to one embodiment of the present disclosure. FIG. 8 is a drawing for explaining the setting of a driving path for sterilizing a plurality of sterilization areas according to one embodiment of the present disclosure. FIG. 9 is a block diagram for specifically explaining the functional configuration of a robot according to one embodiment of the present disclosure. FIG. 10 is a flowchart illustrating a control method according to one embodiment of the present disclosure. Specific details for implementing the invention

[0026] The present disclosure will be described in detail below with reference to the attached drawings.

[0027] The terms used in the embodiments of this disclosure have been selected to be as widely used as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant explanatory section of this disclosure. Therefore, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0028] In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, actions, or components such as parts) and do not exclude the presence of additional features.

[0029] The expression "at least one of A or / and B" should be understood as representing either "A" or "B" or "A and B".

[0030] Expressions such as "first," "second," "first," or "second" used in this disclosure may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0031] Where it is stated that a component (e.g., Component 1) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., Component 2), it should be understood that the component may be directly connected to the other component or connected through the other component (e.g., Component 3).

[0032] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "consisting of" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0033] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of "modules" or a plurality of "parts" may be integrated into at least one module and implemented by at least one processor (not shown), except for a "module" or "part" that needs to be implemented in specific hardware.

[0034] In the present disclosure, the term "user" may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0035] An embodiment of the present disclosure will be described in more detail below with reference to the attached drawings.

[0036] FIG. 1 is a drawing for explaining a method of operating a robot according to one embodiment of the present disclosure.

[0037] Generally, the transmission of a virus can occur through droplets or contact with a contaminated point, but FIG. 1 describes the function of a robot (100) on the premise that it performs a sterilization operation to prevent the transmission of a virus through droplets generated by coughing. A camera (120) equipped in a robot (100) according to one example has a fixed field of view. In this case, the camera (120) cannot detect cough events occurring from all directions. Therefore, if a cough event occurs outside the field of view of the camera (120), the robot (100) cannot identify the sterilization area on its own.

[0038] According to one example, a robot (100) may be equipped with a plurality of microphones (130). When an audio signal corresponding to a cough sound is input through the plurality of microphones (130), the robot (100) can identify the direction in which the audio signal was generated. Subsequently, the robot (100) can control a camera (120) to photograph the direction in which the cough event occurred. In this process, the camera module (not shown) itself may be rotated to photograph the direction in which the cough event occurred, but the robot (100) itself may be rotated using a drive unit (110) provided in the robot (100) to photograph the direction in which the cough event occurred.

[0039] The robot (100) can identify whether there is a user (200) not wearing a mask within the video of the direction in which the coughing event occurred, and move to the location where the user (200) not wearing a mask is located.

[0040] Accordingly, various embodiments will be described below that can identify areas requiring sterilization without the need for separate sensors installed in an indoor space or user operation.

[0041] FIG. 2 is a block diagram for explaining the configuration of a robot according to one embodiment of the present disclosure.

[0042] Referring to FIG. 2, a robot (100) according to one embodiment of the present disclosure may include a driving unit (110), a camera (120), a microphone (130), and a processor (140).

[0043] The drive unit (110) is a device capable of driving the robot (100), and the drive unit (110) can adjust the driving direction and driving speed according to the control of the processor (140). To this end, the drive unit (110) may include a power generation device that generates power for the robot (100) to drive (e.g., a gasoline engine, a diesel engine, an LPG (liquefied petroleum gas) engine, an electric motor, etc., depending on the fuel (or energy source) used), a steering device for adjusting the driving direction (e.g., manual steering, hydraulics steering, electronic control power steering (EPS), etc.), and a driving device that drives the robot (100) according to the power (e.g., wheels, propellers, etc.). Here, the drive unit (110) may be modified according to the driving type of the robot (100) (e.g., wheel type, walking type, flying type, etc.).

[0044] The camera (120) can acquire an image by taking a picture of an area within the camera's field of view (FoV).

[0045] The camera (120) may include a lens that focuses visible light or a signal received by an object, e.g., a user (200), onto an image sensor, and an image sensor capable of detecting visible light or a signal. Here, the image sensor may include a 2D pixel array divided into a plurality of pixels. In one embodiment, the camera (120) may be implemented as a depth camera.

[0046] The microphone (130) is configured to receive an audio signal. The audio signal that can be received through the microphone (130) may be sound in an audible frequency band or an inaudible frequency band. Sound in the audible frequency band has a frequency of sound that humans can hear, and may be sound from 20 Hz to 20 kHz. Additionally, sound in the inaudible frequency band has a frequency of sound that humans cannot hear, and may be sound from 10 kHz to 300 GHz. Here, the audio signal may be sound corresponding to a cough sound, and the microphone (130) according to one example may be composed of multiple microphones arranged in different directions.

[0047] The processor (140) controls the overall operation of the robot (100). Specifically, the processor (140) is connected to each component of the robot (100) to control the overall operation of the robot (100). For example, the processor (140) is connected to the drive unit (110), camera (120), and microphone (130) to control the operation of the robot (100).

[0048] According to one embodiment, the processor (140) may be named by various names such as a digital signal processor (DSP), a microprocessor, a central processing unit (CPU), a Micro Controller Unit (MCU), a micro processing unit (MPU), a Neural Processing Unit (NPU), a controller, or an application processor (AP), but in this specification, it is described as a processor (130). The processor (140) may be implemented as a System on Chip (SoC) or a Large Scale Integration (LSI), or may be implemented in the form of a Field Programmable Gate Array (FPGA). Additionally, the processor (140) may include volatile memory such as SRAM.

[0049] A processor (140) according to one embodiment can identify the direction of generation of an audio signal when an audio signal input through a plurality of microphones is identified as an audio signal corresponding to a cough sound.

[0050] Here, the processor (140) may use a neural network model that can be stored in memory (not shown) or downloaded from an external server (not shown) to identify whether the input audio signal corresponds to a cough sound. Additionally, the audio signal may be identified as corresponding to a cough sound based on the ratio of signals having a specific frequency included in the audio signal input through the microphone (130).

[0051] In order for the processor (140) to identify the direction in which an audio signal corresponding to a cough sound is generated, the microphone (130) may be composed of multiple microphones positioned in different directions. The processor (140) can identify the direction in which the audio signal is generated based on the difference in the strength of the audio signals input to each of the multiple microphones positioned in different directions.

[0052] Next, the processor (140) can control the camera (120) to photograph the direction identified as the direction of the generation of the audio signal corresponding to the cough sound. Specifically, the processor (140) can adjust the orientation of the camera so that the direction of the generation of the audio signal is included within the field of view of the camera (120).

[0053] The processor (140) can identify a sterilization area based on the user's location when a user not wearing a mask is identified in an image acquired through the camera (120).

[0054] According to one example, the processor (140) may use a neural network model that can be stored in memory (not shown) or downloaded from an external server (not shown) to identify whether the acquired image contains a person not wearing a mask. According to another example, the processor (140) may identify whether the image contains a person not wearing a mask by directly analyzing the image to identify an object corresponding to the mask. For example, the processor (140) may identify whether the image contains a person not wearing a mask by identifying an object corresponding to the mask based on various characteristic information included in the image, such as edge information, color information, shape information, etc.

[0055] The processor (140) can identify an area requiring sterilization (hereinafter referred to as a sterilization area) based on the location of a user not wearing a mask. In this specification, when a robot performs a sterilization function to prevent the spread of viruses through droplets, the location of a person not wearing a mask is expressed using the term 'sterilization point'.

[0056] The processor (140) can identify a sterilization area based on a preset range relative to a sterilization point and control the drive unit (110) so that the robot (100) can move to the identified sterilization area.

[0057] According to one example, the processor (140) can identify a preset range based on a sterilization point as a sterilization area based on the strength of the audio signal. For example, the processor (140) can identify a first threshold distance range based on the sterilization point as a sterilization area when the strength of the audio signal is less than a threshold strength, and identify a second threshold distance range greater than the first threshold distance based on the sterilization point as a sterilization area when the strength of the audio signal is greater than or equal to the threshold strength.

[0058] According to another example, the processor (140) may identify a sterilization area based on the number of people not wearing masks included in the image captured through the camera (120). For example, the processor (140) may identify a first threshold distance range based on the sterilization point as a sterilization area when the number of people not wearing masks is less than a threshold number, and identify a second threshold distance range larger than the first threshold distance based on the sterilization point as a sterilization area when the number of people not wearing masks is greater than or equal to the threshold number.

[0059] According to another example, if multiple users who are not wearing masks are identified, the processor (140) can identify multiple sterilization zones based on the location of each of the multiple users.

[0060] Additionally, the processor (140) can determine at least one of the size of the sterilization area or the sterilization intensity based on whether the location identified by a user not wearing a mask belongs to a preset area.

[0061] Here, the preset area may include at least one of an area where an object with a high frequency of contact by users is located, an area with a high frequency of visits by users, or an area with a low frequency of mask wearing by users.

[0062] According to one embodiment of the present disclosure, a processor (140) controls a camera (120) to periodically photograph all directions of the space where the robot (100) is located, and can identify the location distribution of users and the mask-wearing ratio within the images of all directions of the space. Subsequently, the processor (140) can periodically sterilize the indoor space based on the location distribution of users and the mask-wearing ratio.

[0063] Here, the robot (100) can perform a sterilization function every first cycle for at least one of the areas where the location distribution value of users exceeds a threshold value or the area where the ratio of people not wearing masks exceeds a threshold ratio.

[0064] On the other hand, the robot (100) can perform a sterilization function every second period for at least one of the areas where the location distribution value of users is below a threshold value or the area where the ratio of people not wearing masks is below a threshold ratio. Here, the first period may be shorter than the second period.

[0065] According to one example, the processor (140) can use a first neural network model to identify whether an input audio signal corresponds to a cough sound. Here, the first neural network model may be a model trained to identify whether an input audio signal contains a cough sound, and the first neural network model may be stored in memory (not shown) which is a component of the robot (100) or downloaded from an external server (not shown). In this case, the first neural network model may be updated periodically or upon the occurrence of an event.

[0066] Additionally, the processor (140) can identify whether the acquired image includes a user who is not wearing a mask by using a second neural network model. Here, the second neural network model may be a model trained to identify whether the input image includes a user who is not wearing a mask, and the second neural network model may be stored in memory (not shown) which is a component of the robot (100) or downloaded from an external server (not shown). In this case, the first neural network model may be updated periodically or upon the occurrence of an event.

[0067] Additionally, a robot (100) according to one embodiment of the present disclosure further includes a distance sensor (not shown), and a processor (140) can identify the direction and distance of each of the plurality of sterilization areas using the distance sensor when a plurality of sterilization areas are identified, and can set a driving path for sterilization work based on the identified direction and distance. For example, the distance sensor may be implemented as a LiDAR sensor, a depth camera, etc.

[0068] Additionally, a robot (100) according to one embodiment of the present disclosure further includes a sterilization device (not shown), and a processor (140) can control the sterilization device to perform a sterilization function when the robot (100) moves to an identified sterilization area.

[0069] FIGS. 3a and 3b illustrate the size of a sterilization area corresponding to the loudness of a cough sound according to one embodiment of the present disclosure.

[0070] A processor (140) according to one embodiment of the present disclosure can determine the size of a sterilization area based on the intensity of an audio signal corresponding to a cough sound.

[0071] The less the user wears a mask, and the stronger the cough, the louder the cough sound tends to be, and at the same time, the spread of droplets can occur actively. Therefore, the processor (140) can determine the size of the sterilization area to be larger as the strength of the audio signal corresponding to the cough sound increases.

[0072] Specifically, the processor (140) can identify a threshold range area as a sterilization area based on the location of a user not wearing a mask, based on the strength of an audio signal corresponding to a cough sound, and the processor (140) can identify a larger threshold range area as a sterilization area as the strength of the audio signal increases.

[0073] Referring to FIG. 3a, a user (200) may make a faint cough (hereinafter referred to as the first cough) that produces a small sound. A robot (100) according to one example may identify the strength of an audio signal corresponding to the first cough sound and identify a first threshold range area (310) based on the location of the user (200) as a sterilization area.

[0074] Referring to FIG. 3b, the user (200) may cough loudly (hereinafter referred to as the second cough). The second cough may include not only a normal cough but also a sneeze. A robot (100) according to one example may identify the intensity of an audio signal corresponding to the second cough sound and identify the area (320) of the second threshold range based on the location of the user (200) as a sterilization area. Here, the first threshold range may have a value smaller than the second threshold range.

[0075] Although not illustrated in FIGS. 3a and 3b, the processor (140) may identify a sterilization area based not only on the intensity of the audio signal corresponding to the cough sound but also on the frequency of the cough sound included in the audio signal.

[0076] FIGS. 4a and 4b illustrate the size of a sterilization area corresponding to the number of people not wearing a mask according to one embodiment of the present disclosure.

[0077] A processor (140) according to one embodiment of the present disclosure can determine the size of a sterilization area based on the number of users not wearing masks among the users included in the image obtained through the camera (120).

[0078] When multiple users are present within the field of view of the camera (120), the more users who are not wearing masks, the more active the spread of droplets caused by coughing may occur. Additionally, since non-infected individuals may be included among the users who are not wearing masks, the area requiring sterilization needs to be set wider. Therefore, the processor (140) can determine the size of the sterilization area to be larger as the number of users who are not wearing masks increases.

[0079] Referring to FIG. 4a, within the field of view of the camera (120) there may be a user wearing a mask (201) and a user not wearing a mask (202). In this case, the processor (140) can identify the number of users not wearing a mask as one and identify a first sterilization area (410) corresponding thereto.

[0080] Referring to FIG. 4b, there may be multiple users (201, 201) who are not wearing masks. In this case, the processor (140) may identify the number of users not wearing masks as 2 and identify a second sterilization area (420) corresponding thereto. Here, the second sterilization area may include a wider range of areas than the first sterilization area.

[0081] FIGS. 5A and 5B are drawings for explaining a method for determining a sterilization area for sterilizing a plurality of sterilization points according to one embodiment of the present disclosure.

[0082] A processor (140) according to one embodiment of the present disclosure can determine a sterilization area in various ways when a plurality of users (201, 202, 203) who are not wearing masks are identified. Specifically, the processor (140) can identify the location of each person not wearing a mask included in images taken of the direction in which a plurality of cough events were identified as an individual sterilization point, and can identify a sterilization area based on the identified sterilization point.

[0083] Referring to FIG. 5a, a processor (140) according to one example can identify a point (500, hereinafter, reference point) spaced apart by the same distance (d) from a plurality of users (201, 202, 203) who are not wearing masks. To identify a sterilization area that includes all of the plurality of users (201, 202, 203) who are not wearing masks, the processor (140) can identify a circular area with a radius of d+a, centered on the reference point (500), and formed by adding a preset value (a) to the distance (d) from the reference point (500) to each user, as a sterilization area.

[0084] If the number of users not wearing masks exceeds 3, the above-mentioned reference point (500) may be determined based on the first 3 users for whom a cough event was identified.

[0085] Referring to FIG. 5b, a processor (140) according to one example can identify a critical range area as a sterilization area based on the location of each user (201, 202, 203) who is not wearing a mask. For example, the processor (140) can identify a circular area as a sterilization area centered on the location of each user (201, 202, 203) who is not wearing a mask, having a radius d which is the distance from a reference point (500) to each user.

[0086] Here, if a new cough event occurs after the robot (100) has performed a sterilization function on an identified sterilization area, the robot (100) can identify a new reference point (500) and additionally identify a new sterilization area based thereon.

[0087] FIG. 6 illustrates the size and sterilization intensity of a sterilization area corresponding to a location containing a sterilization point according to one embodiment of the present disclosure.

[0088] Specifically, a processor (140) according to one embodiment of the present disclosure can identify the size (620) and sterilization intensity (630) of a sterilization area based on a location (610) containing a sterilization point where a cough event was identified.

[0089] For example, if the sterilization point identified by the processor (140) is a hallway (611), the processor (140) can identify the size (620) of the sterilization area as 'large' because the hallway (611) is a place where users continuously pass through. Meanwhile, the processor (140) can identify the sterilization intensity (630) as 'weak' because the hallway (611) has a low probability of infection through contact with objects and users spend little time at one point.

[0090] As another example, if the sterilization point identified by the processor (140) is a restroom (612), the processor (140) may identify the size of the sterilization area (620) as 'medium' because the restroom (612) is a place where droplets can spread to facilities such as toilets, urinals, and sinks. Meanwhile, the processor (140) may identify the sterilization intensity (630) as 'medium' because the restroom (612) has a somewhat high possibility of infection through contact with objects.

[0091] As another example, if the sterilization point identified by the processor (140) is a door (613), the processor (140) can identify the size (620) of the sterilization area as 'small' because the door (613) blocks droplet spread between the two spaces separated by the door (613). Meanwhile, the processor (140) can identify the sterilization intensity (630) as 'medium' because the door (613) has a somewhat high possibility of infection through contact with objects.

[0092] Finally, if the sterilization point identified by the processor (140) is a drinking fountain (614), the processor (140) can identify the size (620) of the sterilization area as 'small' because only the area included within the critical range from the point where the drinking fountain (614) is located needs to be sterilized. Meanwhile, the processor (140) can identify the sterilization intensity (630) as 'strong' because the drinking fountain (614) includes facilities used by users without wearing masks and the possibility of infection through contact with objects is significantly high. Here, strong, medium, and small are merely classifications according to example, and the size of the sterilization area and the sterilization intensity can be classified into various levels.

[0093] As described above, by determining the size and sterilization intensity of the sterilization area according to various embodiments of the present disclosure, the robot (100) is able to perform an optimal sterilization function corresponding to individual locations.

[0094] FIG. 7a is a diagram illustrating an audio signal processing method using a neural network model according to one embodiment of the present disclosure.

[0095] A processor (140) according to one embodiment of the present disclosure can identify whether an audio signal input through a microphone (130) is an audio signal corresponding to a cough sound using a first neural network model. Specifically, the processor (140) can input the input audio signal (711) into a first neural network model (710) to obtain information (712) related to whether the audio signal includes a cough sound.

[0096] Here, the first neural network model may be a model trained to identify whether the input audio signal (711) contains a cough sound. Additionally, the first neural network model may be a model trained to identify the type of cough sound included in the audio signal and the number of users who produced the cough sound.

[0097] FIG. 7b is a drawing for explaining an image processing method using a neural network model according to one embodiment of the present disclosure.

[0098] A processor (140) according to one embodiment of the present disclosure can identify whether a person not wearing a mask is included in an image acquired through a camera (120) using a second neural network model. Specifically, the processor (140) can input the acquired image (721) into a second neural network model (720) to obtain information (722) related to whether a person not wearing a mask is included in the image.

[0099] Here, the second neural network model may be a model trained to identify whether the acquired image (721) includes a user who is not wearing a mask. Recently, as the demand for masks has increased, masks of various designs are being manufactured and distributed. A processor (140) according to one embodiment of the present disclosure can accurately identify a user wearing a mask and a user not wearing a mask by using the second neural network model (720).

[0100] FIG. 8 is a drawing for explaining the setting of a driving path for sterilizing a plurality of sterilization areas according to one embodiment of the present disclosure.

[0101] A robot (100) according to one embodiment of the present disclosure further includes a distance sensor (160), and a processor (140) according to one example can identify the direction and distance of each of the plurality of sterilization areas (810, 820, 830) using the distance sensor (160) when the plurality of sterilization areas are identified, and can set a driving path (300) for sterilization work based on the identified direction and distance. Here, the distance sensor (160) may be implemented as a LIDAR (Light Detection And Ranging).

[0102] Specifically, the processor (140) can identify the angle measured clockwise from the reference line (800) as the direction to each sterilization area, and can identify the distance from the position of the robot (100) to each sterilization point included in each sterilization area as the distance to each sterilization area.

[0103] Referring to FIG. 8, the processor (140) can identify the direction of the first sterilization area (810) as 80 degrees (811) and the distance as 20 meters (812). Additionally, the processor (140) can identify the direction of the second sterilization area (820) as 120 degrees (821) and the distance as 30 meters (822). Finally, the processor (140) can identify the direction of the third sterilization area (830) as 160 degrees (831) and the distance as 15 meters (832).

[0104] A processor (140) according to one embodiment of the present disclosure can set a driving path according to a preset method based on the direction and distance of the identified first to third sterilization areas. FIG. 8 illustrates that a driving path (300) is set according to a method of preferentially sterilizing a sterilization area having a low value of the angle corresponding to the identified direction for each sterilization area.

[0105] Specifically, the processor (140) can set the driving path (300) of the robot (100) in the order of sterilizing the first sterilization area first because its direction is closest to the reference line (800), then the second sterilization area, and finally the third sterilization area located furthest from the reference line (800).

[0106] However, the driving path setting method described in FIG. 8 is merely an example, and the processor (140) can set the driving path according to any other method. According to various embodiments of the present disclosure, if the robot (100) is a guide bot or retail bot placed in an indoor space where many people are active, there is an advantage of being able to sterilize multiple sterilization areas quickly and efficiently.

[0107] Although not illustrated in FIG. 8, if the robot (100) is a guide bot or a retail bot, the robot (100) may further include a display (not illustrated) including a touch screen, and the robot (100) may sterilize the display itself after a preset time has elapsed following the input of a user's touch operation.

[0108] FIG. 9 is a block diagram for specifically explaining the functional configuration of a robot according to one embodiment of the present disclosure.

[0109] According to FIG. 9, the robot (100') includes a drive unit (110), a camera (120), a microphone (130), a processor (140), a memory (150), a distance sensor (160), and a sterilization device (170). A detailed description of the configurations shown in FIG. 9 that overlap with the configuration shown in FIG. 2 will be omitted.

[0110] The memory (150) can store data necessary for various embodiments of the present disclosure. Depending on the purpose of data storage, the memory (150) may be implemented in the form of a memory embedded in the robot (100) or in the form of a memory that can be attached to and detached from the robot (100). For example, data for driving the robot (100) may be stored in a memory embedded in the robot (100), and data for the expansion function of the robot (100) may be stored in a memory that can be attached to and detached from the robot (100). Meanwhile, the memory embedded in the robot (100) may be implemented as at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD). Additionally, the memory that can be attached to the robot (100) may be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.) or external memory that can be connected to a USB port (e.g., USB memory).

[0111] According to one example, the memory (150) may store at least one instruction or a computer program containing instructions for controlling the robot (100).

[0112] According to another example, the memory (150) may store information regarding a neural network model including multiple layers. Here, storing information regarding a neural network model may mean storing various information related to the operation of the neural network model, such as information about multiple layers included in the neural network model, and information about parameters used in each of the multiple layers (e.g., filter coefficients, biases, etc.). For example, according to one embodiment, the memory (150) may store information regarding a first neural network model (151) trained to identify whether an input audio signal includes a cough sound, and a second neural network model (152) trained to identify whether an acquired image includes a user not wearing a mask.

[0113] The distance sensor (160) is configured to measure the distance between the robot (100) and the sterilization point where a cough event occurred. The distance sensor (160) can be implemented using a LIDAR (Light Detection And Ranging) or a depth camera, etc. According to one example, the distance sensor (160) can measure the distance between the robot (100) and the sterilization point through triangulation, TOF (Time of Flight) measurement, or phase difference displacement measurement.

[0114] The sterilization device (170) performs the function of killing viruses and bacteria when the robot (100) reaches the sterilization area and performs the sterilization function. According to one example, the sterilization device (170) may be composed of a disinfectant spray module (not shown) including a tank in which the disinfectant is stored, a pipe through which the disinfectant flows, and a nozzle part for spraying the disinfectant.

[0115] According to another example, the sterilization device (170) may be implemented as a UV sterilization device including an LED capable of irradiating ultraviolet (UV) light. The sterilization device (170) may not only sterilize the sterilization area but also perform the function of sterilizing a display (not shown) equipped on the robot (100).

[0116] FIG. 10 is a flowchart illustrating a control method according to one embodiment of the present disclosure.

[0117] A control method for a robot according to one embodiment of the present disclosure identifies the direction of generation of an audio signal based on the arrangement direction of each of the plurality of microphones when an audio signal input through a plurality of microphones is identified as an audio signal corresponding to a cough sound (S1010). Subsequently, the identified direction is captured through a camera, and if a user not wearing a mask is identified in the image acquired through the camera, a sterilization area is identified based on the user's location (S1020). Finally, the robot moves to the identified sterilization area (S1030).

[0118] Here, the step of identifying a sterilization area (S1020) can identify a critical range area as a sterilization area based on the strength of the audio signal and the user's location.

[0119] Additionally, the step of identifying the sterilization area (S1020) can determine the size of the sterilization area based on the number of users not wearing masks.

[0120] Additionally, the step of identifying a sterilization area (S1020) can identify a sterilization area based on the location of each of the multiple users when multiple users who are not wearing masks are identified.

[0121] Additionally, the step of identifying a sterilization area (S1020) can determine at least one of the size of the sterilization area or the sterilization intensity based on whether the user identified the location belongs to a preset area.

[0122] Here, the preset area may include at least one of an area where an object with a high frequency of contact by users is located, an area with a high frequency of visits by users, or an area with a low frequency of mask wearing by users.

[0123] Meanwhile, the step of identifying the direction in which the audio signal is generated (S1010) identifies whether the input audio signal corresponds to a cough sound using a first neural network model, and the first neural network model may be a model trained to identify whether the input audio signal includes a cough sound.

[0124] Meanwhile, the step of identifying a sterilization area (S1020) identifies whether the acquired image includes a user who is not wearing a mask using a second neural network model, and the second neural network model may be a model trained to identify whether the acquired image includes a user who is not wearing a mask.

[0125] In addition, when multiple sterilization areas are identified, the method may further include the step of identifying the direction and distance of each of the multiple sterilization areas and the step of setting a driving path for sterilization based on the identified direction and distance.

[0126] Meanwhile, when the robot moves to an identified sterilization area, it may further include a step of performing a sterilization function.

[0127] Meanwhile, the methods according to the various embodiments of the present disclosure described above can be implemented in the form of an application that can be installed on an existing robot.

[0128] In addition, the methods according to the various embodiments of the present disclosure described above can be implemented by software upgrades or hardware upgrades for existing robots alone.

[0129] In addition, the various embodiments of the present disclosure described above may also be performed through an embedded server equipped in a robot, or through an external server of at least one of the robots.

[0130] Meanwhile, the various embodiments described above may be implemented in a recording medium readable by a computer or a similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented in the processor (140) itself. According to the software implementation, embodiments such as the procedures and functions described herein may be implemented in separate software modules. Each of the software modules may perform one or more functions and operations described herein.

[0131] Meanwhile, computer instructions for performing processing operations of a robot (100) according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. When computer instructions stored in such a non-transitory computer-readable medium are executed by a processor of a specific device, they cause the specific device to perform processing operations in the robot (100) according to various embodiments described above.

[0132] A non-transient computer-readable medium refers to a medium that stores data semi-permanently and can be read by a device, unlike media that store data for a short period of time such as registers, caches, and memory. Specific examples of non-transient computer-readable media include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0133] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure. Explanation of the symbols

[0134] 100: Robot 110: Drive unit 120: Camera 130: Microphone 140: Processor

Claims

Claim 1 A robot comprising: a drive unit; a camera; a plurality of microphones arranged in different directions; and a processor that, based on the difference in intensity of audio signals input to each of the plurality of microphones arranged in different directions, identifies the direction of occurrence of the audio signal when the audio signal input through the plurality of microphones is identified as an audio signal corresponding to a cough sound, controls the camera to capture the identified direction of the audio signal corresponding to the cough sound when the audio signal input through the plurality of microphones is identified as the cough sound, and when a user not wearing a mask is identified in the image acquired through the camera, identifies a sterilization area based on the location of the user and controls the drive unit to move to the identified sterilization area. Claim 2 A robot according to claim 1, wherein the processor identifies a critical range area as the sterilization area based on the position of a user not wearing the mask, based on the strength of the audio signal. Claim 3 In claim 1, the processor determines the size of the sterilization area based on the number of users not wearing the mask, a robot. Claim 4 In claim 1, the processor is a robot that identifies the sterilization area based on the location of each of the plurality of users when a plurality of users not wearing the mask are identified. Claim 5 In claim 1, the processor is a robot that determines at least one of the size of the sterilization area or the sterilization intensity based on whether the location of the user not wearing the mask is identified as belonging to a preset area. Claim 6 In paragraph 5, the above-mentioned preset area includes at least one of an area where an object with a high frequency of contact by users is located, an area with a high frequency of visits by users, or an area with a low frequency of mask wearing by users, a robot Claim 7 A robot according to claim 1, wherein the processor identifies whether the input audio signal is an audio signal corresponding to the cough sound using a first neural network model, and the first neural network model is a model trained to identify whether the input audio signal includes a cough sound. Claim 8 A robot according to claim 1, wherein the processor identifies whether the acquired image includes a user not wearing a mask using a second neural network model, and the second neural network model is a model trained to identify whether the acquired image includes a user not wearing a mask. Claim 9 A robot according to claim 1, further comprising a distance sensor; wherein the processor, when a plurality of sterilization areas are identified, identifies the direction and distance of each of the plurality of sterilization areas using the distance sensor, and sets a driving path for a sterilization operation based on the identified direction and distance. Claim 10 A robot according to claim 1, further comprising a sterilization device that performs a sterilization function; wherein the processor controls the sterilization device to perform a sterilization function when the robot moves to the identified sterilization area. Claim 11 A control method for a robot comprising a plurality of microphones and a camera arranged in different directions, comprising: a step of identifying the direction of generation of the audio signal based on the arrangement direction of each of the plurality of microphones when the audio signal input through the plurality of microphones is identified as an audio signal corresponding to a cough sound based on the difference in intensity of the audio signal input through each of the plurality of microphones arranged in different directions; a step of capturing the identified direction of the audio signal corresponding to the cough sound when the audio signal input through the plurality of microphones is identified as the cough sound through the camera, and when a user not wearing a mask is identified in the image obtained through the camera, a step of identifying a sterilization area based on the location of the user; and a step of moving to the identified sterilization area. Claim 12 In claim 11, the step of identifying the sterilization area is a control method that identifies a critical range area as a sterilization area based on the position of a user not wearing the mask, based on the intensity of the audio signal. Claim 13 In claim 11, the step of identifying the sterilization area is a control method that determines the size of the sterilization area based on the number of users not wearing the mask. Claim 14 In claim 11, the step of identifying the sterilization area is a control method in which, when a plurality of users not wearing the mask are identified, the sterilization area is identified based on the location of each of the plurality of users. Claim 15 In claim 11, the step of identifying the sterilization area is a control method that determines at least one of the size of the sterilization area or the sterilization intensity based on whether the identified location of a user not wearing the mask belongs to a preset area. Claim 16 A control method according to claim 15, wherein the above-mentioned preset area includes at least one of an area where an object with a high frequency of contact by users is located, an area with a high frequency of visits by users, or an area with a low frequency of mask wearing by users. Claim 17 In claim 11, the step of identifying the direction in which the audio signal is generated is a control method in which the input audio signal is identified using a first neural network model as an audio signal corresponding to the cough sound, and the first neural network model is a model trained to identify whether the input audio signal includes the cough sound. Claim 18 A control method according to claim 11, wherein the step of identifying the sterilization area is to identify whether the acquired image includes a user not wearing a mask using a second neural network model, and the second neural network model is a model trained to identify whether the acquired image includes a user not wearing a mask. Claim 19 A control method according to claim 11, further comprising: a step of identifying the direction and distance of each of the plurality of sterilization areas when a plurality of sterilization areas are identified; and a step of setting a driving path for a sterilization operation based on the identified direction and distance. Claim 20 A control method according to claim 11, further comprising the step of performing a sterilization function when the robot moves to the identified sterilization area.

Images