Electronic apparatus and control method thereof
By determining the target location with the longest reverberation time within a candidate area, the electronic device improves voice recognition accuracy and user satisfaction, addressing the challenges of varying voice quality and complex environments.
Patent Information
- Application Number
- PCT/KR2024/018523
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-12-18
- Filing Date
- 2024-11-21
- Publication Date
- 2025-06-26
AI Technical Summary
Existing electronic devices face challenges in accurately recognizing user voice commands due to varying voice quality, distance from the device, and complex or obstructed spaces, leading to low recognition rates and user dissatisfaction.
The electronic device identifies a candidate area with a threshold size in map data, moves to a representative location, outputs an audio signal, and determines the reverberation time at multiple locations to identify a target location with the longest reverberation time, optimizing voice recognition.
This approach enhances voice recognition accuracy by determining the optimal location for receiving user voice commands, improving user satisfaction and device functionality in various environments.
Smart Images

Figure KR2024018523_26062025_PF_FP_ABST
Abstract
Description
Electronic device and method of controlling the same
[0001] The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device that recognizes a user command and performs a function corresponding to the user command, and a control method thereof.
[0002] Electronic devices may include a microphone to recognize user commands in a specific space. When receiving user commands through a microphone, the recognition rate may vary depending on the quality of the user's voice, including the user command.
[0003] When a user speaks from a distance, recognition rates may be low depending on the location of the electronic device. Furthermore, voice recognition rates may be low if the electronic device is located in a complex space or surrounded by obstacles.
[0004] For mobile electronic devices, the device may spend significant time waiting in its charging location. While the device is waiting in its charging location, the user or the user's voice can trigger or activate the device. If the user's voice recognition is low in the charging location, it will be difficult to accurately respond to user commands, resulting in lower user satisfaction.
[0005] There is a need to determine where the user's voice can be best recognized in the space where electronic devices are placed. If the user arbitrarily selects the location, determining the most effective location can be difficult.
[0006] The present disclosure is designed to improve the above-described problem, and an object of the present disclosure is to provide an electronic device and a control method thereof that provide a target location for receiving a user's voice by taking into account the reverberation time of an audio signal.
[0007] According to one embodiment, an electronic device includes a speaker, a microphone, and at least one processor that identifies a candidate area having a threshold area or greater in map data related to a space in which the electronic device is located, moves to a representative location of the candidate area and outputs an audio signal through the speaker, obtains a reverberation time of the audio signal based on a recorded audio signal corresponding to the audio signal at a plurality of locations including the representative location within the candidate area obtained through the microphone, and identifies a location at the plurality of locations having the longest reverberation time as a target location.
[0008] The at least one processor can identify an area having a threshold area or more among a plurality of areas in the map data as a candidate area, and identify a target location in the candidate area.
[0009] The at least one processor may obtain a perimeter length of a first region having a threshold area or greater, obtain a length corresponding to an open space in the first region, and divide the length corresponding to the open space by the perimeter length, and if the ratio of the open space is less than the threshold ratio, identify the first region as a candidate region.
[0010] The at least one processor may, after outputting the audio signal, obtain a first volume of the audio signal at a first point in time based on the recorded audio signal, obtain a second volume obtained by multiplying the first volume by a preset reverberation ratio, obtain a second point in time at which the audio signal becomes the second volume, and obtain the reverberation time based on a difference between the first point in time and the second point in time.
[0011] The at least one processor can identify a location where the reverberation time is greater than or equal to a threshold time among the plurality of locations as a candidate location, and identify the target location based on the candidate location and additional information.
[0012] The additional information includes a power supply location for connecting power to the electronic device, and the at least one processor can identify a candidate location closest to the power supply location among a plurality of candidate locations where a distance difference between the candidate location and the power supply location is less than a threshold distance, as the target location.
[0013] The above additional information includes a candidate projection surface location associated with the output of the projection image, and the at least one processor can identify a candidate location closest to the candidate projection surface location as the target location.
[0014] The recorded audio signal is a first recorded audio signal, the target location is a first target location, the at least one processor identifies an utterance location of a user command during a threshold period, outputs the audio signal based on the utterance location through the speaker, obtains a second recorded audio signal including the audio signal through the microphone, obtains a reverberation time of the audio signal based on the second recorded audio signal, and identifies a location having the longest reverberation time among the plurality of locations as a second target location, and the second target location may be different from the first target location.
[0015] The electronic device further includes a display, and the at least one processor can control the display to display a UI including the map data indicating the target location.
[0016] The at least one processor may further include a step of moving to the target location when a preset event occurs.
[0017] According to one embodiment, a method of controlling an electronic device includes the steps of: identifying a candidate area having a threshold area or greater in map data related to a space where the electronic device is located; moving to a representative location of the candidate area and outputting an audio signal; obtaining a recorded audio signal corresponding to the audio signal at a plurality of locations within the candidate area; obtaining a reverberation time of the audio signal at the plurality of locations including the representative location within the candidate area based on the recorded audio signal; and identifying a location having the longest reverberation time at the plurality of locations as a target location.
[0018] The step of identifying the candidate area may identify an area having a threshold area or larger among a plurality of areas in the map data as a candidate area, and the step of identifying the target location may identify a target location in the candidate area.
[0019] The step of identifying the candidate region may include obtaining a perimeter length of a first region having a threshold area or greater, obtaining a length corresponding to an open space in the first region, and dividing the length corresponding to the open space by the perimeter length, and if the ratio of the open space is less than the threshold ratio, identifying the first region as a candidate region.
[0020] The step of obtaining the reverberation time may include, after outputting the audio signal, obtaining a first volume of the audio signal at a first point in time based on the recorded audio signal, obtaining a second volume obtained by multiplying the first volume by a preset reverberation ratio, obtaining a second point in time at which the audio signal becomes the second volume, and obtaining the reverberation time based on a difference between the first point in time and the second point in time.
[0021] The step of identifying the target location may include identifying a location among the plurality of locations where the reverberation time is greater than or equal to a threshold time as a candidate location, and identifying the target location based on the candidate location and additional information.
[0022] The above-mentioned additional information includes a power supply location for connecting power to the electronic device, and the step of identifying the target location can identify a candidate location that is closest to the power supply location among a plurality of candidate locations where a distance difference between the candidate location and the power supply location is less than a threshold distance as the target location.
[0023] The above additional information includes a candidate projection surface location related to the output of the projection image, and the step of identifying the target location can identify a candidate location closest to the candidate projection surface location as the target location.
[0024] The above-mentioned recorded audio signal is a first recorded audio signal, the target location is a first target location, the control method further includes a step of identifying an utterance location of a user command during a threshold period, a step of outputting the audio signal based on the utterance location, a step of obtaining a second recorded audio signal including the audio signal, a step of obtaining a reverberation time of the audio signal based on the second recorded audio signal, and a step of identifying a location having the longest reverberation time among the plurality of locations as a second target location, wherein the second target location may be different from the first target location.
[0025] The above control method may further include a step of displaying a UI including the map data indicating the target location.
[0026] The above control method may further include a step of moving to the target location when a preset event occurs.
[0027] FIG. 1 is a diagram illustrating an operation of determining a target location corresponding to an electronic device according to one embodiment.
[0028] FIG. 2 is a block diagram illustrating an electronic device according to one embodiment.
[0029] FIG. 3 is a block diagram illustrating a specific configuration of the electronic device of FIG. 2, according to one embodiment.
[0030] FIG. 4 is a diagram illustrating an operation of determining a representative location of a space in which an electronic device is located, according to one embodiment.
[0031] FIG. 5 is a drawing for explaining an operation of determining a target location in a space where an electronic device is located, according to one embodiment.
[0032] FIG. 6 is a drawing for explaining an operation of determining a target position according to one embodiment.
[0033] FIG. 7 is a diagram for explaining an operation of determining a target position by analyzing a reverberation time according to one embodiment.
[0034] FIG. 8 is a diagram for explaining an operation of determining a candidate area by considering a critical area according to one embodiment.
[0035] FIG. 9 is a diagram illustrating an operation of determining a candidate area using a ratio of the length of an open space, according to one embodiment.
[0036] FIG. 10 is a diagram illustrating an operation of determining a candidate area using a ratio of a critical area and an open space length, according to one embodiment.
[0037] FIG. 11 is a diagram for explaining an operation of obtaining a reverberation time according to one embodiment.
[0038] Fig. 12 is a diagram for explaining the reverberation time according to one embodiment.
[0039] FIG. 13 is a diagram for explaining an operation of determining a target position using a reverberation time according to one embodiment.
[0040] FIG. 14 is a drawing for explaining an operation of determining a target location by considering a power supply location according to one embodiment.
[0041] FIG. 15 is a drawing for explaining an operation of determining a target position by considering a candidate projection surface position according to one embodiment.
[0042] FIG. 16 is a drawing for explaining an operation of determining a target position by considering a power supply possible position and a candidate projection surface position, according to one embodiment.
[0043] FIG. 17 is a drawing for explaining a screen that provides a target location according to one embodiment.
[0044] FIG. 18 is a drawing for explaining a screen for selecting a target location according to one embodiment.
[0045] FIG. 19 is a drawing for explaining a screen related to setting a target location, according to one embodiment.
[0046] FIG. 20 is a diagram for explaining an operation of determining a target location by considering a user's speech history according to one embodiment.
[0047] FIG. 21 is a diagram for explaining an operation of outputting a test audio signal by taking into account the utterance location according to one embodiment.
[0048] FIG. 22 is a drawing for explaining an operation of determining a target position by considering an ignition position according to one embodiment.
[0049] FIG. 23 is a drawing for explaining a control method of an electronic device according to one embodiment.
[0050] Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.
[0051] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.
[0052] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.
[0053] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".
[0054] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.
[0055] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).
[0056] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.
[0057] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented as at least one processor (not shown), excluding any "modules" or "parts" that need to be implemented as specific hardware.
[0058] In this specification, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).
[0059] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.
[0060] FIG. 1 is a drawing for explaining an operation of determining a target position corresponding to an electronic device (100) according to one embodiment.
[0061] The electronic device (100) may be a mobile device. The electronic device (100) may include the movable member (122) of FIG. 3. The electronic device (100) may be a device that can move via the movable member (122). For example, the electronic device (100) may be implemented as a mobile projector or a mobile robot. The mobile robot may represent a cleaning robot or a service robot. Referring to the embodiment (1000) of FIG. 1, the electronic device (100) may determine a target location in a specific space. The target location may indicate a location suitable for receiving a user's voice. The electronic device (100) may receive the user's voice via a microphone. The user's voice may be clearly recognized depending on the location of the electronic device (100). The electronic device (100) may move to the target location to easily recognize the user's voice in a specific space.
[0062] A target location may include at least one of a charging location, a standby location, and a placement location. The charging location may indicate a location for charging an electronic device (100). The charging location may be a location where a charging device (or charging station) for charging the electronic device (100) is placed. For example, the charging location may be a location where a charging device for charging a device requiring charging, such as a robot or a cordless vacuum cleaner, is placed.
[0063] The standby position may be a standby position for receiving a user command. The electronic device (100) may standby at a specific position in a charging mode or a standby mode that does not perform the basic functions of the electronic device (100). The electronic device (100) may obtain a user command in the standby position. For example, the robot may move to the standby position after performing a user command. The electronic device (100) may wait in the standby position until a user command is received. The mode may be described as a state or a method.
[0064] The placement location may be a location where the electronic device (100) is placed by the user. For example, if the user must place the electronic device (100) directly, the placement location may be provided. For example, the user may be provided with a placement location for placing a fixed device, such as an AI speaker including a microphone.
[0065] FIG. 2 is a block diagram illustrating an electronic device (100) according to one embodiment.
[0066] Referring to FIG. 2, the electronic device (100) may include at least one of a memory (113), at least one processor (111), a display, a speaker (117), and a microphone (118).
[0067] The electronic device (100) may be an electronic whiteboard, TV, desktop PC, laptop, smartphone, tablet PC, server, etc. The above-described examples are merely examples for describing the electronic device and are not necessarily limited to the above-described devices.
[0068] At least one processor (111) can perform overall control operations of the electronic device (100). At least one processor (111) can perform a function of controlling overall operations of the electronic device (100).
[0069] The memory (113) can store a test audio signal. The memory (113) can store map data related to the space where the electronic device (100) is located. The test audio signal can be described as an audio signal or an output audio signal.
[0070] The speaker (117) can output a test audio signal.
[0071] A microphone (118) can acquire a recording audio signal including a test audio signal. The recording audio signal can include at least one of analog data or converted digital data.
[0072] At least one processor (111) may obtain map data related to a space where an electronic device (100) is located, identify a candidate area having a threshold area or more in the map data, determine a representative position representing the candidate area, output a test audio signal stored in a memory (113) based on the representative position through a speaker (117), obtain a recorded audio signal including the test audio signal through a microphone (118), obtain a reverberation time of the test audio signal based on the recorded audio signal, determine a position in the candidate area with the longest reverberation time as a target position, and provide the target position.
[0073] At least one processor (111) may obtain map data related to a space in which an electronic device (100) is located. The map data may include data related to a space in which the electronic device (100) is placed or a space in which the electronic device (100) is to be driven. The map data may be defined as two-dimensional coordinates or three-dimensional coordinates. The map data may be described as map information, spatial information, spatial data, map characteristic data, spatial coordinates, etc.
[0074] At least one processor (111) can distinguish multiple areas from map data. At least one processor (111) can distinguish individual spaces from map data into a single area. At least one processor (111) can identify multiple areas representing individual spaces from map data. At least one processor (111) can obtain (or calculate) an area for each of the multiple areas. At least one processor (111) can identify (or determine) an area having an area greater than a threshold area as a candidate area.
[0075] At least one processor (111) can identify a representative location representing a candidate area. For example, the representative location may be the center point of the candidate area. The representative location may be described as a representative coordinate, a reference location, or a reference coordinate. For example, one candidate area may have only one representative location. For example, one candidate area may have multiple representative locations.
[0076] At least one processor (111) can output a test audio signal through a speaker (117) based on a representative location.
[0077] According to one embodiment, at least one processor (111) can be fixed at a representative position and output a test audio signal.
[0078] According to one embodiment, at least one processor (111) may output a test audio signal at a representative location and continuously output the test audio signal while moving within the candidate area.
[0079] The test audio signal may be an audio signal previously stored in memory (113), etc. The test audio signal may be a signal output to measure reverberation time. The test audio signal needs to be clearly distinguished from the analysis target. The test audio signal may include a specific frequency or a specific waveform.
[0080] At least one processor (111) can obtain a recording audio signal including a test audio signal output through a microphone (118). The recording audio signal can be described as audio data or audio information. The recording audio signal can include sounds surrounding the electronic device (100). At least one processor (111) can identify the test audio signal from the recording audio signal.
[0081] Depending on the recording time through the microphone (118), the test audio signal may be direct sound or undirect sound. Undirect sound may mean a signal that has been reflected at least once.
[0082] At least one processor (111) can analyze a test audio signal included in a recorded audio signal to obtain a reverberation time. At least one processor (111) can determine a target location based on the reverberation time.
[0083] Reverberation time can be defined as the time it takes for an audio signal to decay to a critical rate relative to its initial output volume (or loudness). A longer reverberation time may result in less energy loss. A longer reverberation time may also result in a more recognizable sound.
[0084] At least one processor (111) can divide map data into multiple areas, determine an area having a threshold area or larger among the multiple areas as a candidate area, and determine a target location in the candidate area.
[0085] The critical area can be changed according to the user's settings. At least one processor (111) can identify each space as a single area based on map data. At least one processor (111) can distinguish multiple areas based on map data. At least one processor (111) can determine an area having a critical area among the multiple areas as a candidate area. The candidate area can be described as a filtered area or a filtered area.
[0086] The operation of determining a candidate area using a critical area is described in Fig. 8.
[0087] At least one processor (111) may obtain a perimeter length of a first region (specific region) having a critical area or greater, obtain a length corresponding to an open space in the first region (specific region), obtain a ratio of the open space by dividing the length corresponding to the open space by the perimeter length, and if the ratio of the open space is less than the critical ratio, determine the first region (specific region) as a candidate region.
[0088] A specific area can be classified as either a closed space or an open space. A closed space can represent an immovable space, such as a wall. An open space can represent a movable space, such as a doorway. A closed space can be described by a closed section or a line corresponding to the closed section. An open space can be described by an open section or a line corresponding to the open section.
[0089] At least one processor (111) can obtain a perimeter length that includes both closed and open spaces in a specific area. The perimeter length can be described as a border length. At least one processor (111) can obtain a length corresponding to an open space.
[0090] At least one processor (111) can obtain ratio information based on the perimeter length and the length corresponding to the open space. At least one processor (111) can obtain ratio information (or ratio value) by dividing the length corresponding to the open space by the perimeter length. The ratio information can refer to the ratio occupied by the open space in a specific area. At least one processor (111) can determine an area where the ratio information is greater than a threshold ratio as a candidate area.
[0091] According to various embodiments, the operation of determining a candidate region using a critical area and a critical ratio is described in FIG. 10.
[0092] According to various embodiments, the operation of determining a candidate region using a threshold ratio is described in FIG. 9.
[0093] At least one processor (111) can output a test audio signal, obtain a first volume of the test audio signal at a first point in time based on a recorded audio signal, obtain a second volume by multiplying the first volume by a preset reverberation ratio, obtain a second point in time at which the volume of the test audio signal becomes the second volume, and obtain a reverberation time based on a difference between the first point in time and the second point in time. A description related to this is described in FIGS. 11 and 12.
[0094] At least one processor (111) can determine a location in a candidate region where the reverberation time is greater than or equal to a threshold time as a candidate location, and determine a target location based on the candidate location and additional information. A description related to this is provided in FIG. 13.
[0095] The additional information includes a power supply location for connecting power to the electronic device (100), and at least one processor (111) can obtain a distance difference between a candidate location and a power supply location, and determine a candidate location closest to the power supply location among a plurality of candidate locations with a distance difference less than a threshold distance as a target location. A description related to this is provided in FIG. 14.
[0096] The additional information includes a candidate projection surface location associated with the output of the projection image, and at least one processor (111) can determine the candidate location closest to the candidate projection surface location as the target location. A description related to this is described in FIG. 15.
[0097] The recorded audio signal may be a first recorded audio signal, and the target location may be the first target location. At least one processor (111) identifies an utterance location of a user command during a threshold period, outputs a test audio signal based on the utterance location through a speaker (117), obtains a second recorded audio signal including the test audio signal through a microphone (118), obtains a reverberation time of the test audio signal based on the second recorded audio signal, and determines a location in the candidate area with the longest reverberation time as a second target location, and the second target location may be different from the first target location. Descriptions related to the utterance location are described in FIGS. 20, 21, and 22.
[0098] The electronic device (100) may further include a display. At least one processor (111) may control the display to display a UI including map data indicating a target location. A description thereof is provided in FIGS. 17, 18, and 19.
[0099] At least one processor (111) can move to a target location when a preset event occurs.
[0100] The preset event may include at least one of an event in which a control command for moving the electronic device (100) is received, an event in which a wake-up word for calling the electronic device (100) is recognized, an event in which the charging power is below a threshold, and an event in which recognition of a user's voice fails.
[0101] The electronic device (100) can determine a target location with a relatively long reverberation time using a test audio signal. The target location can represent a location where, on average, the audio signal is well recognized.
[0102] Target locations allow users to easily identify the location where audio signals are best heard. Considering the location where audio signals are best heard, users can determine charging, standby, and placement locations.
[0103] According to various embodiments, the electronic device (100) may be a mobile device. For example, the electronic device (100) may be a mobile robot. For example, the electronic device (100) may be a mobile cleaning robot. For example, the electronic device (100) may be a mobile projector.
[0104] The electronic device (100) may include a movable member. The movable member may be controlled by a driving unit. The electronic device (100) may transmit power generated by a motor to the movable member through the driving unit. The electronic device (100) may move to a specific location through the movable member.
[0105] When the electronic device (100) is implemented as a projector, the electronic device (100) may include a projection unit for outputting a projected image.
[0106] FIG. 3 is a block diagram for explaining a specific configuration of the electronic device (100) of FIG. 2, according to one embodiment.
[0107] Referring to FIG. 3, the electronic device (100) may include at least one of a processor (111), a projection unit (112), a memory (113), a communication interface (114), an operation interface (115), an input / output interface (116), a speaker (117), a microphone (118), a power supply unit (119), a driving unit (120), a sensor unit (121), or a moving member (122).
[0108] The configuration illustrated in FIG. 3 is merely an example of various embodiments, and some configurations may be omitted and new configurations may be added.
[0109] The content already explained in Fig. 2 is omitted.
[0110] At least one processor (111) may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) that processes a digital signal. However, the present invention is not limited thereto, and may include one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a graphics-processing unit (GPU), a communication processor (CP), or an advanced reduced instruction set computer (RISC) machines (ARM) processor, or may be defined by the relevant terminology. At least one processor (111) may be implemented as a system on chip (SoC) having a built-in processing algorithm, a large scale integration (LSI), or may be implemented in the form of a field programmable gate array (FPGA). At least one processor (111) may perform various functions by executing computer executable instructions stored in a memory (113).
[0111] The projection unit (112) is a component that projects an image to the outside. According to various embodiments of the present disclosure, the projection unit (112) can be implemented in various projection methods (e.g., CRT (cathode-ray tube) method, LCD (Liquid Crystal Display) method, DLP (Digital Light Processing) method, laser method, etc.). For example, the CRT method has the same principle as a CRT monitor. The CRT method magnifies the image with a lens in front of the cathode-ray tube (CRT) and displays the image on the screen. Depending on the number of cathode-ray tubes, it is divided into a single-tube type and a three-tube type, and in the case of a three-tube type, the red, green, and blue cathode-ray tubes can be implemented separately.
[0112] Another example is the LCD method, which displays images by passing light from a light source through liquid crystals. LCD methods are divided into single-panel and three-panel types. In the case of the three-panel type, light from a light source is separated into red, green, and blue by a dichroic mirror (a mirror that reflects only a specific color of light and transmits all others). After passing through the liquid crystals, the light can be refocused into a single point.
[0113] Another example is the DLP method, which displays images using a DMD (Digital Micromirror Device) chip. The DLP projection unit may include a light source, a color wheel, a DMD chip, a projection lens, etc. The light output from the light source can be colored as it passes through the rotating color wheel. The light passing through the color wheel is input to the DMD chip. The DMD chip contains numerous micromirrors and reflects the light input to the DMD chip. The projection lens can play a role in magnifying the light reflected from the DMD chip to the image size.
[0114] Another example is a laser system that uses a Diode Pumped Solid State (DPSS) laser and a galvanometer. A multi-color laser uses three DPSS lasers, one for each RGB color, with their optical axes overlapped by a special mirror. The galvanometer includes a mirror and a high-power motor that moves the mirror at high speeds. For example, the galvanometer can rotate the mirror at up to 40 kHz / sec. The galvanometer is mounted according to the scan direction, but since projectors typically scan in a planar manner, the galvanometer can also be positioned separately along the x and y axes.
[0115] The projection unit (112) may include various types of light sources. For example, the projection unit (112) may include at least one light source among a lamp, an LED, and a laser.
[0116] The projection unit (112) can output images in a 4:3 screen ratio, a 5:4 screen ratio, or a 16:9 wide screen ratio depending on the purpose of the electronic device (100) or the user's settings, and can output images in various resolutions such as WVGA (854*480), SVGA (800*600), XGA (1024*768), WXGA (1280*720), WXGA (1280*800), SXGA (1280*1024), UXGA (1600*1200), and Full HD (1920*1080) depending on the screen ratio.
[0117] The projection unit (112) can perform various functions for adjusting the output image under the control of at least one processor (111). For example, the projection unit (112) can perform functions such as zoom, keystone, quick corner (4 corner) keystone, and lens shift.
[0118] Specifically, the projection unit (112) can enlarge or reduce the image depending on the distance from the screen (projection distance). That is, the zoom function can be performed depending on the distance from the screen. At this time, the zoom function may include a hardware method of adjusting the screen size by moving the lens and a software method of adjusting the screen size by cropping the image, etc. When the zoom function is performed, the focus of the image needs to be adjusted. For example, the method of adjusting the focus includes a manual focus method, an electric focus method, etc. The manual focus method refers to a method of focusing manually, and the electric focus method refers to a method of automatically focusing using a motor built into the projector when the zoom function is performed. When performing the zoom function, the projection unit (112) may provide a digital zoom function through software, and may provide an optical zoom function of performing the zoom function by moving the lens through the driving unit (120).
[0119] The projection unit (112) can perform a keystone correction function. If the height is not right for front projection, the screen may be distorted upwards or downwards. The keystone correction function refers to a function that corrects a distorted screen. For example, if distortion occurs in the left and right directions of the screen, it can be corrected using horizontal keystone, and if distortion occurs in the up and down directions, it can be corrected using vertical keystone. The quick corner (4 corner) keystone correction function is a function that corrects the screen when the center area of the screen is normal but the corner areas are not balanced. The lens shift function is a function that moves the screen as it is when the screen is off the screen.
[0120] The projection unit (112) can automatically analyze the surrounding environment and projection environment without user input to provide zoom / keystone / focus functions. Specifically, the projection unit (112) can automatically provide zoom / keystone / focus functions based on the distance between the electronic device (100) and the screen detected by a sensor (depth camera, distance sensor, infrared sensor, light sensor, etc.), information about the space where the electronic device (100) is currently located, information about the amount of ambient light, etc.
[0121] The projection unit (112) can provide a lighting function using a light source. In particular, the projection unit (112) can provide a lighting function by outputting a light source using an LED. According to various embodiments, the projection unit (112) can include one LED, and according to other embodiments, the electronic device (100) can include a plurality of LEDs. The projection unit (112) can output a light source using a surface-emitting LED according to an implementation example. The surface-emitting LED can refer to an LED having a structure in which an optical sheet is arranged on the upper side of the LED so that the light source is evenly distributed and output. Specifically, when a light source is output through the LED, the light source can be evenly distributed through the optical sheet, and the light source distributed through the optical sheet can be incident on the display panel.
[0122] The projection unit (112) can provide the user with a dimming function for adjusting the intensity of the light source. Specifically, when a user input for adjusting the intensity of the light source is received from the user through the operation interface (115) (e.g., a touch display button or dial), the projection unit (112) can control the LED to output the intensity of the light source corresponding to the received user input.
[0123] The projection unit (112) can provide a dimming function based on content analyzed by at least one processor (111) without user input. Specifically, the projection unit (112) can control the LED to output the intensity of a light source based on information about the currently provided content (e.g., content type, content brightness, etc.).
[0124] The projection unit (112) can control the color temperature under the control of at least one processor (111). The at least one processor (111) can control the color temperature based on the content. Specifically, when the content is identified to be output, the at least one processor (111) can obtain frame-by-frame color information of the content whose output has been determined. Then, the at least one processor (111) can control the color temperature based on the obtained frame-by-frame color information. The at least one processor (111) can obtain at least one primary color of the frame based on the frame-by-frame color information. Then, the at least one processor (111) can adjust the color temperature based on the obtained at least one primary color. For example, the color temperature that the at least one processor (111) can adjust can be classified into a warm type or a cold type. It is assumed that a frame to be output (hereinafter, referred to as an output frame) includes a scene in which a fire has occurred. At least one processor (111) can identify (or obtain) that the primary color is red based on color information included in the current output frame. In addition, at least one processor (111) can identify a color temperature corresponding to the identified primary color (red). The color temperature corresponding to red may be a warm type. At least one processor (111) can use an artificial intelligence model to obtain the color information or primary color of the frame. According to various embodiments, the artificial intelligence model can be stored in the electronic device (100) (e.g., the memory (113)). According to another embodiment, the artificial intelligence model can be stored in an external server that can communicate with the electronic device (100).
[0125] The memory (113) may be implemented as an internal memory such as a ROM (e.g., an electrically erasable programmable read-only memory (EEPROM)) or RAM included in at least one processor (111), or may be implemented as a separate memory from at least one processor (111). In this case, the memory (113) may be implemented as a memory embedded in the electronic device (100) or as a memory detachable from the electronic device (100) depending on the purpose of data storage. For example, data for driving the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for expanding functions of the electronic device (100) may be stored in a memory detachable from the electronic device (100).
[0126] In the case of memory embedded in the electronic device (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), etc.), hard drive, or solid state drive (SSD), and in the case of memory that can be attached or detached to the electronic device (100), it may be implemented in the form of a memory card (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC), etc.), external memory that can be connected to a USB port (e.g., USB memory), etc.
[0127] The memory (113) may store at least one command regarding the electronic device (100). In addition, the memory (113) may store an O / S (Operating System) for driving the electronic device (100). The memory (113) may also store various software programs or applications for operating the electronic device (100) according to various embodiments of the present disclosure. In addition, the memory (113) may include a semiconductor memory such as a flash memory or a magnetic storage medium such as a hard disk.
[0128] Specifically, various software modules for operating the electronic device (100) according to various embodiments of the present disclosure may be stored in the memory (113), and at least one processor (111) may control the operation of the electronic device (100) by executing various software modules stored in the memory (113). That is, the memory (113) is accessed by at least one processor (111), and data reading / recording / modifying / deleting / updating, etc. may be performed by at least one processor (111).
[0129] In the present disclosure, the term memory (113) may be used to mean a storage unit, a ROM, a RAM within at least one processor (111), or a memory card (e.g., a micro SD card, a memory stick) mounted on an electronic device (100).
[0130] The communication interface (114) is a configuration that performs communication with various types of external devices according to various types of communication methods. The communication interface (114) may include a wireless communication module or a wired communication module. Each communication module may be implemented in the form of at least one hardware chip.
[0131] A wireless communication module may be a module that communicates wirelessly with an external device. For example, the wireless communication module may include at least one of a Wi-Fi module, a Bluetooth module, an infrared communication module, or other communication modules.
[0132] Wi-Fi and Bluetooth modules can communicate via Wi-Fi and Bluetooth, respectively. When using a Wi-Fi or Bluetooth module, various connection information, such as the service set identifier (SSID) and session key, is first transmitted and received. This information is then used to establish a communication connection before various other information can be transmitted and received.
[0133] Infrared communication modules perform communication based on infrared communication (IrDA, infrared Data Association) technology, which transmits data wirelessly over short distances using infrared light, which is between visible light and millimeter waves.
[0134] In addition to the above-described communication method, other communication modules may include at least one communication chip that performs communication according to various wireless communication standards such as zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.
[0135] A wired communication module may be a module that communicates with an external device via a wire. For example, the wired communication module may include at least one of a Local Area Network (LAN) module, an Ethernet module, a paired cable, a coaxial cable, a fiber optic cable, or an Ultra Wide-Band (UWB) module.
[0136] The manipulation interface (115) may include various types of input devices. For example, the manipulation interface (115) may include a physical button. In this case, the physical button may include a function key, a directional key (e.g., a four-way key), or a dial button. According to various embodiments, the physical button may be implemented as multiple keys. According to another embodiment, the physical button may be implemented as one key. When the physical button is implemented as one key, the electronic device (100) may receive a user input in which one key is pressed for a threshold time or longer. When a user input in which one key is pressed for a threshold time or longer is received, at least one processor (111) may perform a function corresponding to the user input. For example, at least one processor (111) may provide a lighting function based on the user input.
[0137] The manipulation interface (115) can receive user input using a non-contact method. When receiving user input using a contact method, physical force must be transmitted to the electronic device (100). Therefore, a method for controlling the electronic device (100) regardless of physical force may be required. Specifically, the manipulation interface (115) can receive user gestures and perform operations corresponding to the received user gestures. The manipulation interface (115) can receive user gestures through a sensor (e.g., an image sensor or an infrared sensor).
[0138] The manipulation interface (115) can receive user input using a touch method. For example, the manipulation interface (115) can receive user input via a touch sensor. According to various embodiments, the touch method can be implemented in a non-contact manner. For example, the touch sensor can determine whether the user's body has approached within a threshold distance. The touch sensor can identify user input even when the user does not touch the touch sensor. According to another implementation example, the touch sensor can identify user input when the user touches the touch sensor.
[0139] The electronic device (100) can receive user input in various ways other than the above-described operation interface (115). In various embodiments, the electronic device (100) can receive user input through an external remote control device. The external remote control device can be a remote control device corresponding to the electronic device (100) (e.g., a dedicated control device of the electronic device (100)) or a user's portable communication device (e.g., a smartphone or wearable device). The user's portable communication device can store an application for controlling the electronic device (100). The portable communication device can obtain user input through the stored application and transmit the obtained user input to the electronic device (100). The electronic device (100) can receive user input from the portable communication device and perform an operation corresponding to the user's control command.
[0140] The electronic device (100) can receive user input using voice recognition. According to various embodiments, the electronic device (100) can receive the user's voice through a microphone included in the electronic device (100). According to other embodiments, the electronic device (100) can receive the user's voice from a microphone or an external device. Specifically, the external device can acquire the user's voice through the microphone of the external device and transmit the acquired user's voice to the electronic device (100). The user's voice transmitted from the external device can be audio data or digital data converted from audio data (e.g., audio data converted into a frequency domain, etc.). The electronic device (100) can perform an operation corresponding to the received user's voice. Specifically, the electronic device (100) can receive audio data corresponding to the user's voice through the microphone. In addition, the electronic device (100) can convert the received audio data into digital data. In addition, the electronic device (100) can convert the converted digital data into text data using the STT (Speech To Text) function. According to various embodiments, the STT (Speech To Text) function can be performed directly in the electronic device (100).
[0141] According to another embodiment, the STT (Speech To Text) function may be performed by an external server. The electronic device (100) may transmit digital data to the external server. The external server may convert the digital data into text data and obtain control command data based on the converted text data. The external server may transmit the control command data (which may also include text data) to the electronic device (100). The electronic device (100) may perform an operation corresponding to the user's voice based on the obtained control command data.
[0142] The electronic device (100) may provide a voice recognition function using a single assistant (or artificial intelligence assistant, e.g., Bixby), but this is merely an example and the electronic device (100) may provide the voice recognition function using multiple assistants. In this case, the electronic device (100) may provide the voice recognition function by selecting one of the multiple assistants based on a trigger word corresponding to the assistant or a specific key present on the remote control.
[0143] The electronic device (100) can receive user input using screen interaction. Screen interaction may refer to a function of identifying whether a predetermined event occurs through an image projected by the electronic device (100) on the screen (or projection surface) and acquiring user input based on the predetermined event. The predetermined event may refer to an event in which a predetermined object is identified at a specific location (e.g., a location where a UI for receiving user input is projected). The predetermined object may include at least one of a part of the user's body (e.g., a finger), a pointer, or a laser point. If the predetermined object is identified at a location corresponding to the projected UI, the electronic device (100) can identify that a user input for selecting the projected UI has been received. For example, the electronic device (100) may project a guide image to display the UI on the screen. In addition, the electronic device (100) can identify whether the user selects the projected UI. Specifically, the electronic device (100) can identify that the user has selected the projected UI when a predetermined event is identified at a location of the projected UI. The projected UI can include at least one item. The electronic device (100) can perform spatial analysis to identify whether the predetermined event is at the location of the projected UI. The electronic device (100) can perform spatial analysis through a sensor (e.g., an image sensor, an infrared sensor, a depth camera, a distance sensor, etc.). By performing the spatial analysis, the electronic device (100) can identify whether the predetermined event occurs at a specific location (a location where the UI is projected). In addition, when the predetermined event is identified as occurring at a specific location (a location where the UI is projected), the electronic device (100) can identify that a user input for selecting a UI corresponding to the specific location has been received.
[0144] The input / output interface (116) is configured to input / output at least one of an audio signal and an image signal. The input / output interface (116) can receive at least one of an audio signal and an image signal from an external device, and can output a control command to the external device.
[0145] Depending on the implementation example, the input / output interface (116) may be implemented as an interface that inputs / outputs only audio signals and an interface that inputs / outputs only image signals, or may be implemented as one interface that inputs / outputs both audio signals and image signals.
[0146] In various embodiments of the present disclosure, the input / output interface (116) may be implemented as at least one wired input / output interface among HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), USB C-type, DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (Dsubminiature), and DVI (Digital Visual Interface). According to various embodiments, the wired input / output interface may be implemented as an interface that inputs / outputs only audio signals and an interface that inputs / outputs only image signals, or may be implemented as one interface that inputs / outputs both audio signals and image signals.
[0147] The electronic device (100) can receive data via a wired input / output interface, but this is merely an example of various embodiments, and can also receive power via the wired input / output interface. For example, the electronic device (100) can receive power from an external battery via a USB C-type or from an outlet via a power adapter. As another example, the electronic device (100) can receive power from an external device (e.g., a laptop or monitor) via a DP.
[0148] The audio signal may be implemented to be input through a wired input / output interface, and the image signal may be implemented to be input through a wireless input / output interface (or communication interface). Alternatively, the audio signal may be implemented to be input through a wireless input / output interface (or communication interface), and the image signal may be implemented to be input through a wired input / output interface.
[0149] The speaker (117) is a component that outputs an audio signal. In particular, the speaker (117) may include an audio output mixer, an audio signal processor, and an audio output module. The audio output mixer may synthesize a plurality of audio signals to be output into at least one audio signal. For example, the audio output mixer may synthesize an analog audio signal and another analog audio signal (e.g., an analog audio signal received from the outside) into at least one analog audio signal. The audio output module may include a speaker or an output terminal. According to various embodiments, the audio output module may include a plurality of speakers, and in this case, the audio output module may be arranged inside the main body, and sound emitted by covering at least a portion of the diaphragm of the audio output module may pass through a waveguide and be transmitted to the outside of the main body. The audio output module may include a plurality of audio output units, and the plurality of audio output units may be arranged symmetrically on the exterior of the main body, thereby radiating sound in all directions, that is, in a 360-degree omnidirectional manner.
[0150] The microphone (118) is a component for receiving a user's voice or other sounds and converting them into audio data. The microphone (118) can receive the user's voice in an activated state. For example, the microphone (118) can be formed integrally on the upper side, the front side, the side side, etc. of the electronic device (100). The microphone (118) can include various components such as a microphone for collecting the user's voice in analog form, an amplifier circuit for amplifying the collected user's voice, an A / D conversion circuit for sampling the amplified user's voice and converting it into a digital signal, and a filter circuit for removing noise components from the converted digital signal.
[0151] The power supply unit (119) can receive power from an external source and supply power to various components of the electronic device (100). The power supply unit (119) according to various embodiments of the present disclosure can receive power through various methods. In various embodiments, the power supply unit (119) can receive power using a connector. The power supply unit (119) can receive power using a 220 V DC power cord. However, the present invention is not limited thereto, and the electronic device (100) can receive power using a USB power cord or receive power using a wireless charging method.
[0152] The power supply unit (119) can be supplied with power using an internal battery or an external battery. The power supply unit (119) according to various embodiments of the present disclosure can be supplied with power through the internal battery. For example, the power supply unit (119) can charge the power of the internal battery using at least one of a 220V DC power cord, a USB power cord, and a USB C-Type power cord, and can be supplied with power through the charged internal battery. The power supply unit (119) according to various embodiments of the present disclosure can be supplied with power through an external battery. For example, when the electronic device (100) is connected to the external battery through various wired communication methods such as a USB power cord, a USB C-Type power cord, and a socket home, the power supply unit (119) can be supplied with power through the external battery. That is, the power supply unit (119) can be supplied with power directly from the external battery, or can charge the internal battery through the external battery and be supplied with power from the charged internal battery.
[0153] The power supply unit (119) according to the present disclosure can receive power using at least one of the multiple power supply methods described above.
[0154] With respect to power consumption, the electronic device (100) may have a power consumption value (e.g., 43 W) or lower due to socket type and other standards. In this case, the electronic device (100) may vary its power consumption to reduce power consumption when using a battery. That is, the electronic device (100) may vary its power consumption based on the power supply method, power usage, etc.
[0155] The driving unit (120) can drive at least one hardware component included in the electronic device (100). The driving unit (120) can generate a physical force and transmit it to at least one hardware component included in the electronic device (100).
[0156] The driving unit (120) can generate driving power for movement of a hardware component included in the electronic device (100) (e.g., movement of the electronic device (100)) or rotation of the component (e.g., rotation of a projection lens).
[0157] The driving unit (120) can adjust the projection angle of the projection unit (112). The driving unit (120) can move the position of the electronic device (100). The driving unit (120) can control a moving member to move the electronic device (100). For example, the driving unit (120) can control the moving member using a motor.
[0158] The sensor unit (121) may include at least one sensor. Specifically, the sensor unit (121) may include at least one of a tilt sensor for sensing the tilt of the electronic device (100) and an image sensor for capturing an image. The tilt sensor may be an acceleration sensor or a gyro sensor, and the image sensor may mean a camera or a depth camera. The tilt sensor may be described as a motion sensor. The sensor unit (121) may include various sensors in addition to the tilt sensor or the image sensor. For example, the sensor unit (121) may include an illuminance sensor and a distance sensor. The distance sensor may be a ToF (Time of Flight). The sensor unit (121) may include a lidar sensor.
[0159] The electronic device (100) can control a lighting function by linking with an external device. Specifically, the electronic device (100) can receive lighting information from the external device. The lighting information can include at least one of brightness information or color temperature information set in the external device. The external device can refer to a device connected to the same network as the electronic device (100) (e.g., an IoT device included in the same home / work network) or a device that is not in the same network as the electronic device (100) but can communicate with the electronic device (100) (e.g., a remote control server). For example, assume that an external lighting device (IoT device) included in the same network as the electronic device (100) is outputting red light at a brightness of 50. The external lighting device (IoT device) can directly or indirectly transmit lighting information (e.g., information indicating that it is outputting red light at a brightness of 50) to the electronic device (100). The electronic device (100) can control the output of a light source based on the lighting information received from the external lighting device. For example, if the lighting information received from the external lighting device includes information to output red light at a brightness of 50, the electronic device (100) can output red light at a brightness of 50.
[0160] The electronic device (100) can control a lighting function based on biometric information. Specifically, at least one processor (111) can obtain the user's biometric information. The biometric information can include at least one of the user's body temperature, heart rate, blood pressure, respiration, and electrocardiogram. The biometric information can include various types of information in addition to the information described above. For example, the electronic device (100) can include a sensor for measuring biometric information. The at least one processor (111) can obtain the user's biometric information through the sensor and control the output of a light source based on the obtained biometric information. As another example, the at least one processor (111) can receive the biometric information from an external device through an input / output interface (116). The external device can refer to the user's portable communication device (e.g., a smartphone or a wearable device). The at least one processor (111) can obtain the user's biometric information from the external device and control the output of the light source based on the obtained biometric information. According to an implementation example, the electronic device (100) can identify whether the user is sleeping, and if the user is identified as sleeping (or preparing to sleep), at least one processor (111) can control the output of the light source based on the user's biometric information.
[0161] An electronic device (100) according to various embodiments of the present disclosure can provide various smart functions.
[0162] Specifically, the electronic device (100) is connected to a portable terminal device for controlling the electronic device (100), and a screen output from the electronic device (100) can be controlled through user input input from the portable terminal device. As an example, the portable terminal device can be implemented as a smartphone including a touch display, and the electronic device (100) receives screen data provided by the portable terminal device from the portable terminal device and outputs it, and a screen output from the electronic device (100) can be controlled according to user input input from the portable terminal device.
[0163] The electronic device (100) can share content or music provided by the mobile terminal device by connecting to the mobile terminal device through various communication methods such as Miracast, Airplay, wireless DEX, and Remote PC.
[0164] In addition, the mobile terminal device and the electronic device (100) can be connected in various connection methods. In various embodiments, the mobile terminal device can search for the electronic device (100) to perform a wireless connection, or the electronic device (100) can search for the mobile terminal device to perform a wireless connection. In addition, the electronic device (100) can output content provided by the mobile terminal device.
[0165] In various embodiments, when a mobile terminal device is placed near an electronic device (100) while specific content or music is being output from the mobile terminal device, and a preset gesture is detected through the display of the mobile terminal device (e.g., motion tap view), the electronic device (100) can output the content or music being output from the mobile terminal device.
[0166] In various embodiments, when the mobile terminal device is outputting specific content or music and the mobile terminal device comes closer to the electronic device (100) to a preset distance or less (e.g., non-contact tap view) or the mobile terminal device comes into contact with the electronic device (100) twice at a short interval (e.g., contact tap view), the electronic device (100) can output the content or music being output by the mobile terminal device.
[0167] In the above-described embodiment, it has been described that the same screen as the screen provided by the mobile terminal device is provided by the electronic device (100), but the present disclosure is not limited thereto. That is, when a connection is established between the mobile terminal device and the electronic device (100), the mobile terminal device may output a first screen provided by the mobile terminal device, and the electronic device (100) may output a second screen provided by the mobile terminal device that is different from the first screen. For example, the first screen may be a screen provided by a first application installed on the mobile terminal device, and the second screen may be a screen provided by a second application installed on the mobile terminal device. For example, the first screen and the second screen may be different screens provided by a single application installed on the mobile terminal device. For example, the first screen may be a screen including a remote control-type UI for controlling the second screen.
[0168] An electronic device (100) according to the present disclosure can output a standby screen. For example, if the electronic device (100) is not connected to an external device or if no input is received from the external device for a preset period of time, the electronic device (100) can output a standby screen. Conditions for the electronic device (100) to output a standby screen are not limited to the examples described above, and the standby screen can be output under various conditions.
[0169] The electronic device (100) may output a standby screen in the form of a blue screen, but the present disclosure is not limited thereto. For example, the electronic device (100) may extract only the shape of a specific object from data received from an external device, acquire an amorphous object, and output a standby screen including the acquired amorphous object.
[0170] The electronic device (100) may further include a display.
[0171] The display may be implemented in various forms, such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display panel (PDP), etc. The display may also include a driving circuit, a backlight unit, etc., which may be implemented in forms, such as an amorphous silicon thin film transistor (a-si TFT), a low temperature poly silicon (LTPS) TFT, and an organic TFT (OTFT). The display may be implemented in a touch screen combined with a touch sensor, a flexible display, a three-dimensional display (3D display), etc. According to various embodiments of the present disclosure, the display may include a display panel that outputs an image, as well as a bezel that houses the display panel. In particular, according to various embodiments of the present disclosure, the bezel may include a touch sensor for detecting user interaction.
[0172] The electronic device (100) may further include a shutter unit.
[0173] The shutter portion may include at least one of a shutter, a fixing member, a rail, or a body.
[0174] The shutter can block light output from the projection unit (112). The fixing member can fix the position of the shutter. The rail can be a path for moving the shutter and the fixing member. The body can be configured to include the shutter and the fixing member.
[0175] The movable member (122) may refer to a member for moving from a first position to a second position in a space where the electronic device (100) is placed. The electronic device (100) may control the movable member (122) to move the electronic device (100) using a force generated by the driving unit (120). The electronic device (100) may generate a force to be transmitted to the movable member (122) using a motor included in the driving unit (120).
[0176] The movable member (122) may include at least one wheel (e.g., a circular wheel). The electronic device (100) may move to a target location (or target position) through the movable member. When a user input or a control command is received, the electronic device (100) may rotate the movable member by transmitting a force generated through a motor to the movable member. The electronic device (100) may control the movable member to adjust the rotation speed, rotation direction, etc. The electronic device (100) may perform a movement operation (or movement function) by controlling the movable member based on the target location or the direction of movement, etc.
[0177] FIG. 4 is a drawing for explaining an operation of determining a representative location of a space where an electronic device (100) is located, according to one embodiment.
[0178] The embodiment (410) of FIG. 4 may represent a space in which an electronic device (100) is placed. The electronic device (100) may obtain map data related to the space in which the electronic device (100) is placed.
[0179] According to the embodiment (420) of FIG. 4, a plurality of areas can be indicated in a space in which the electronic device (100) is placed. The electronic device (100) can identify (or distinguish) a plurality of areas (421, 422, 423, 424, 425) in the space in which it is placed.
[0180] According to the embodiment (430) of FIG. 4, the electronic device (100) can identify representative locations (431, 432, 433, 434, 435) for each of a plurality of regions (421, 422, 423, 424, 425). The representative locations may include locations that represent (or represent) each region. The electronic device (100) can identify representative locations for each of the plurality of regions (421, 422, 423, 424, 425). For example, there may be one representative location for each region. For example, there may be multiple representative locations for each region. For example, the representative location may be the center point of each region.
[0181] FIG. 5 is a drawing for explaining an operation of determining a target location in a space where an electronic device (100) is located, according to one embodiment.
[0182] According to the embodiment (510) of FIG. 5, the electronic device (100) can output a test audio signal in each region. The electronic device (100) can output the test audio signal in each region by considering a representative position.
[0183] According to the embodiment (520) of FIG. 5, the electronic device (100) can obtain a recorded audio signal including an output test audio signal. The electronic device (100) can analyze the audio signal to obtain a reverberation time. The electronic device (100) can analyze the reverberation time to determine candidate locations (521, 522, 523).
[0184] According to the embodiment (530) of FIG. 5, the electronic device (100) can determine a target location (540) using additional information among candidate locations (521, 522, 523). For example, the additional information can include locations (531, 532, 533, 534, 535) of candidate projection surfaces. The electronic device (100) can determine a target location (540) that is closest on average to the locations (531, 532, 533, 534, 535) of candidate projection surfaces among the candidate locations (521, 522, 523).
[0185] FIG. 6 is a drawing for explaining an operation of determining a target position according to one embodiment.
[0186] Referring to FIG. 6, the electronic device (100) can determine a representative location of a candidate area from map data (S605). The map data may include data related to the space where the electronic device (100) is located. For example, the electronic device (100) may have map data stored in memory (113). For example, the electronic device (100) may generate map data. The electronic device (100) may sense the space where it is currently located to generate map data.
[0187] The electronic device (100) can identify candidate areas in map data. The candidate areas may be areas representing multiple spaces included in the map data. The candidate areas may be areas representing individual spaces. For example, assume that there are five rooms in the space where the electronic device (100) is placed. The candidate areas may be areas for each of the five rooms.
[0188] The electronic device (100) can specify a location representing a candidate area. The electronic device (100) can identify a representative location representing the candidate area. For example, the representative location may be the center point of the candidate area.
[0189] The electronic device (100) can output a test audio signal at a representative location (S610). The electronic device (100) can output the test audio signal through a speaker.
[0190] The electronic device (100) can determine a target location by analyzing the reverberation time of the output test audio signal (S615). The electronic device (100) can determine the reverberation time using the energy change of the test audio signal and determine the target location using the determined reverberation time.
[0191] FIG. 7 is a diagram for explaining an operation of determining a target position by analyzing a reverberation time according to one embodiment.
[0192] Referring to FIG. 7, the electronic device (100) can obtain map data (S710). The electronic device (100) can identify (or determine or obtain) a candidate area from the map data (S720). The electronic device (100) can determine a representative location of the candidate area (S730). A description related to this is provided in FIG. 6.
[0193] The electronic device (100) can output a test audio signal based on a representative location (S740). The electronic device (100) can determine the location at which to output the test audio signal based on the representative location. The electronic device (100) can output the test audio signal through a speaker (117).
[0194] For example, the electronic device (100) can output a test audio signal at a representative location.
[0195] For example, the electronic device (100) may output a test audio signal at a representative location and output the test audio signal while moving within a candidate area corresponding to the representative location. The electronic device (100) may continuously output the test audio signal.
[0196] The electronic device (100) can obtain a recording audio signal including a test audio signal (S750). The electronic device (100) can obtain the recording audio signal (or audio data) through a microphone (118). The obtained recording audio signal may include the test audio signal.
[0197] The electronic device (100) can obtain a reverberation time based on a recorded audio signal (S760). The test audio signal may bounce off a wall in a space and be reflected at least once. Each time it is reflected, the audio signal may be absorbed, thereby reducing the size of the audio signal. The electronic device (100) can obtain the size of the test audio signal using the recorded audio signal. The electronic device (100) can obtain the reverberation time based on the size of the test audio signal. A description related to this is provided in FIG. 12.
[0198] The electronic device (100) can determine a target location by analyzing the reverberation time (S770). The longer the reverberation time, the higher the recognition rate of the audio signal. The electronic device (100) can determine the target location with the highest recognition rate of the user's voice by considering the reverberation time.
[0199] FIG. 8 is a diagram for explaining an operation of determining a candidate area by considering a critical area according to one embodiment.
[0200] Referring to FIG. 8, the electronic device (100) can obtain map data corresponding to the current location (S810). Step S810 may correspond to step S710 of FIG. 7. For example, based on a preset event, the electronic device (100) can obtain pre-stored map data. For example, based on a preset event, the electronic device (100) can generate map data representing spatial information. The electronic device (100) can sense the surrounding environment and generate map data representing spatial information.
[0201] The electronic device (100) can distinguish (or identify or classify) at least one area from the map data (S821). The at least one area may represent a space surrounded by walls. For example, if the space where the electronic device (100) is located has five rooms, the electronic device (100) can distinguish five areas.
[0202] The electronic device (100) can obtain an area for each of at least one region (S822). The area can be described as area information. The electronic device (100) can calculate the area for each region.
[0203] The electronic device (100) can identify whether an area having an area greater than a threshold area exists (S823). If an area having an area greater than a threshold area exists (S823-Y), the electronic device (100) can determine the area as a candidate area (S829).
[0204] The electronic device (100) can identify whether each of the at least one area acquired in step S821 is greater than or equal to a threshold area. The electronic device (100) can determine an area greater than or equal to a threshold area among the at least one area as a candidate area.
[0205] If there is no area having an area greater than the threshold area, the electronic device (100) can identify that no candidate area exists.
[0206] Depending on the characteristics of the space, areas with excessively small areas may be identified. The electronic device (100) may determine candidate areas by considering the critical area to omit unnecessary calculation processes.
[0207] FIG. 9 is a diagram illustrating an operation of determining a candidate area using a ratio of the length of an open space, according to one embodiment.
[0208] Steps S910 and S921 of FIG. 9 may correspond to steps S810 and S821 of FIG. 8.
[0209] The electronic device (100) can obtain the perimeter length for each of at least one area (S924). The perimeter length may represent the total length of the perimeter surrounding the area. For example, the perimeter length of a square may represent the sum of the lengths of the four sides forming the square.
[0210] The electronic device (100) can obtain the length of an open space for each of at least one region (S925). The regions can be divided into spaces surrounded by walls and spaces not surrounded by walls. A space surrounded by walls can represent a closed space. A space not surrounded by walls can represent an open space.
[0211] A closed space may be a space in which an electronic device (100) cannot move.
[0212] The open space may be a space in which an electronic device (100) can move.
[0213] The perimeter of a region may include lines corresponding to closed spaces and lines corresponding to open spaces.
[0214] The perimeter of a region can be the sum of the lengths of the lines corresponding to the closed space and the lengths of the lines corresponding to the open space.
[0215] For example, suppose there is a single door in a rectangular room. The total perimeter of the rectangular room can be the sum of the length of the door and the width of the walls excluding the door.
[0216] The electronic device (100) can obtain a ratio of the length of the open space to the perimeter length (S926). The ratio can be described as ratio information. The electronic device (100) can obtain the ratio based on the perimeter length of the region and the length of the open space. The electronic device (100) can obtain the ratio (or ratio value) by dividing the perimeter length of the region by the length of the open space. The electronic device (100) can obtain the ratio of the length of the open space to the total perimeter length of the region.
[0217] The electronic device (100) can identify whether the ratio is below a threshold ratio (S927). A higher ratio indicates a relatively higher proportion of open space. If the ratio indicated by the open space is high, the space can be recognized as open.
[0218] If the ratio is less than the critical ratio (S927-Y), the area can be determined as a candidate area (S929).
[0219] If the ratio is not less than the threshold ratio (S927-N), the electronic device (100) can identify that the candidate region does not exist.
[0220] FIG. 10 is a diagram illustrating an operation of determining a candidate area using a ratio of a critical area and an open space length, according to one embodiment.
[0221] Steps S1010, S1021, S1022, and S1023 of FIG. 10 may correspond to steps S810, S821, S822, and S823 of FIG. 8. Duplicate explanations are omitted.
[0222] Steps S1024, S1025, S1026, S1027, and S1029 of FIG. 10 may correspond to steps S924, S925, S926, S927, and S929 of FIG. 9. Duplicate explanation is omitted.
[0223] The electronic device (100) can identify whether the identified area is larger than a threshold area in step S1021. If the identified area is larger than the threshold area, the electronic device (100) can obtain a perimeter length for the identified area (S1024). The electronic device (100) can identify the length of an open space in the identified area (S1025). The electronic device (100) can obtain a ratio by dividing the perimeter length by the length of the open space (S1026).
[0224] The electronic device (100) can identify whether the ratio is greater than or equal to a threshold ratio (S1027). If the ratio is less than the threshold ratio, the electronic device (100) can determine the identified area as a candidate area (S1029).
[0225] FIG. 11 is a diagram for explaining an operation of obtaining a reverberation time according to one embodiment.
[0226] Referring to Fig. 11, the operations of Fig. 11 may correspond to step S760 of Fig. 7. Duplicate explanation is omitted.
[0227] The electronic device (100) can identify a test audio signal (S1161). The electronic device (100) can obtain a test audio signal to be output through a speaker (117).
[0228] The electronic device (100) can move to a representative position of the candidate area (S1162). The electronic device (100) can output a test audio signal based on the representative position (S1163). The electronic device (100) can output the test audio signal through the speaker (117).
[0229] The electronic device (100) can obtain a recording audio signal including a test audio signal (S1164). The electronic device (100) can collect (or obtain) an ambient audio signal through a microphone (118).
[0230] The electronic device (100) can obtain a first volume of a test audio signal at a first point in time based on the recorded audio signal (S1165). The first point in time may correspond to a point in time at which the test audio signal is output. The first point in time may be a threshold time (e.g., 0.5 seconds) from the point in time at which the test audio signal is output. The electronic device (100) can identify the magnitude of the test audio signal included in the recorded audio signal at the first point in time. The magnitude of the signal obtained at the first point in time may correspond to direct sound. The direct sound may correspond to a direct wave. The volume of the test audio signal obtained at the first point in time may represent an initial volume or an original volume.
[0231] The electronic device (100) can identify a second volume obtained by multiplying the first volume by a reverberation ratio (S1166). The reverberation ratio may be a preset ratio. The reverberation ratio may be a ratio that serves as a reference for indicating a reverberation time. For example, the reverberation ratio may be 20%. The reverberation time may refer to the time it takes for an audio signal to decrease by 20% from its initial volume. The reverberation ratio may be changed according to a user's setting. The electronic device (100) can obtain the second volume by multiplying the first volume by the preset reverberation ratio.
[0232] Test audio signals can lose energy over time due to air, walls, and other factors. Each time the signal reflects off a wall, its energy can decrease. As this energy decreases, the volume of the test audio signal can decrease. As time passes, the number of times the audio is reflected off various surfaces, such as walls, increases, which can also reduce the audio's energy.
[0233] The electronic device (100) can obtain a second point in time when the test audio signal becomes the second volume (S1167). The electronic device (100) can analyze the recorded audio signal to determine whether the volume of the test audio signal becomes the second volume. The electronic device (100) can analyze the recorded audio signal to determine the point in time when the test audio signal becomes the second volume as the second point in time.
[0234] The electronic device (100) can obtain (or calculate) the reverberation time based on the difference between the first time point and the second time point (S1168).
[0235] Fig. 12 is a diagram for explaining the reverberation time according to one embodiment.
[0236] According to an embodiment (1210) of FIG. 12, a situation is shown where a test audio signal is output in a first space (1211). The electronic device (100) can output a test audio signal of a first volume (V1) at a first time point (t1). The electronic device (100) can obtain a recording audio signal to obtain the volume of the test audio signal.
[0237] Referring to Table (1212), the electronic device (100) can obtain the volume of the test audio signal based on the recorded audio signal. The electronic device (100) can obtain the second volume (V2) by multiplying the first volume (V1) by the reverberation ratio. The electronic device (100) can obtain the second time point (t2) at which the volume of the test audio signal becomes the second volume (V2). The electronic device (100) can determine the difference between the first time point (t1) and the second time point (t2) as the reverberation time.
[0238] According to the embodiment (1220) of Fig. 12, a situation is shown where a test audio signal is output in a second space (1221). The electronic device (100) can output a test audio signal of a first volume (V1) at a first time point (t1). The electronic device (100) can obtain a recording audio signal to obtain the volume of the test audio signal.
[0239] Referring to Table (1222), the electronic device (100) can obtain the volume of the test audio signal based on the recorded audio signal. The electronic device (100) can obtain the second volume (V2) by multiplying the first volume (V1) by the reverberation ratio. The electronic device (100) can obtain the third time point (t3) at which the volume of the test audio signal becomes the second volume (V2). The electronic device (100) can determine the difference between the first time point (t1) and the third time point (t3) as the reverberation time.
[0240] The area of the space (1221) of embodiment (1220) may be larger than the area of the space (1211) of embodiment (1210). A larger space may increase the time it takes for an audio signal to reflect and return. In an ideal situation, the same percentage of energy may be lost when an audio signal is reflected by a wall or the like. The reverberation time obtained in space (1221) may be longer than the reverberation time obtained in space (1211). In similar environments, a larger space may result in a longer reverberation time.
[0241] FIG. 13 is a diagram for explaining an operation of determining a target position using a reverberation time according to one embodiment.
[0242] Referring to FIG. 13, the electronic device (100) can obtain reverberation times at multiple locations (S1360). Step S1360 may correspond to step S760 of FIG. 7. Duplicate descriptions are omitted.
[0243] The electronic device (100) can identify whether the reverberation time is greater than or equal to a threshold time (S1371). If the reverberation time is not greater than or equal to the threshold time (S1371-N), the electronic device (100) may not determine the location where the reverberation time was measured as a candidate location.
[0244] If the reverberation time is greater than or equal to the threshold time (S1371-Y), the electronic device (100) may determine a location where the reverberation time is greater than or equal to the threshold time as a candidate location (S1372). The electronic device (100) may identify an edge location among locations where the reverberation time is measured in a specific area. The electronic device (100) may determine an edge location where the reverberation time is greater than or equal to the threshold time among a plurality of edge locations as a candidate location. The edge location may be the location of a line indicating the perimeter of the area.
[0245] Candidate locations may be filtered locations from all locations where reverberation times have been measured. By determining some filtered locations, rather than all locations, as candidate locations, the electronic device (100) can simplify the calculation process in determining the target location.
[0246] The electronic device (100) can determine a target location among candidate locations based on additional information (S1373). The additional information may include an outlet location, a candidate projection surface location, an ignition location, etc. Operations related to the outlet location are described in FIG. 14. A description of the candidate projection surface location is described in FIG. 15. A description of the ignition location is described in FIG. 22.
[0247] FIG. 14 is a drawing for explaining an operation of determining a target location by considering a power supply location according to one embodiment.
[0248] Steps S1460, S1471, and S1472 of Fig. 14 may correspond to steps S1360, S1371, and S1372 of Fig. 13. Duplicate explanation is omitted.
[0249] Once the candidate location is determined, the electronic device (100) can obtain a power-supplyable location in the candidate area (S1473). The power-supplyable location may include a location of an outlet from which power can be supplied.
[0250] The electronic device (100) can obtain the distance difference between the candidate location and the power supply location (S1474). The electronic device (100) can identify whether the distance difference is less than a threshold distance (S1475).
[0251] If the distance difference is less than the threshold distance (S1475-Y), the electronic device (100) can determine the location closest to the power supply location among the candidate locations as the target location (S1476).
[0252] When there are multiple candidate locations, the electronic device (100) can determine whether each of the multiple candidate locations is within a threshold distance from a power-supplyable location. The electronic device (100) can determine the candidate location closest to the power-supplyable location among the multiple candidate locations as the target location.
[0253] When there are multiple power supply locations, the electronic device (100) can obtain the distance difference between the power supply location and the candidate location based on each power supply location. The electronic device (100) can determine the candidate location with the lowest average distance difference as the target location.
[0254] For example, the power supply locations may be different for each candidate area.
[0255] For example, there may be multiple power supply locations in a candidate area.
[0256] FIG. 15 is a drawing for explaining an operation of determining a target position by considering a candidate projection surface position according to one embodiment.
[0257] Steps S1560, S1571, and S1572 of FIG. 15 may correspond to steps S1360, S1371, and S1372 of FIG. 13. Duplicate descriptions are omitted. In the embodiment of FIG. 15, the electronic device (100) may be a device (e.g., a projector) that performs a projection function for outputting a projected image.
[0258] Once the candidate location is determined, the electronic device (100) can identify a candidate projection surface in the candidate area (S1577). The candidate projection surface may represent a projection surface on which a projection image can be output. The electronic device (100) may determine a wall surface within the candidate area that has a threshold area or greater as the candidate projection surface. The candidate projection surface may be a portion of a single wall surface. This is because obstacles (e.g., a clock, a wardrobe, etc.) may be placed on the wall surface.
[0259] The electronic device (100) can identify the location of the candidate projection surface (S1578). The electronic device (100) can identify the location of the candidate projection surface in the map data. The electronic device (100) can map the location of the candidate projection surface in the map data.
[0260] The electronic device (100) can determine the position closest to the candidate projection surface position among the candidate positions as the target position (S1579).
[0261] When there are multiple candidate locations, the electronic device (100) can determine the candidate location closest to the candidate projection surface location among the multiple candidate locations as the target location.
[0262] When there are multiple candidate projection surfaces, the electronic device (100) can obtain the distance difference between the candidate projection surface position and the candidate position based on each candidate projection surface. The electronic device (100) can determine the candidate position with the lowest average distance difference as the target position.
[0263] For example, the candidate projection surface location may be different for each candidate area.
[0264] For example, there may be multiple candidate projection surface locations in the candidate region.
[0265] FIG. 16 is a drawing for explaining an operation of determining a target position by considering a power supply possible position and a candidate projection surface position according to one embodiment.
[0266] Steps S1672, S1673, S1674, and S1675 of FIG. 16 may correspond to steps S1472, S1473, S1474, and S1475 of FIG. 14. Duplicate explanations are omitted.
[0267] Steps S1676, S1677, S1678, and S1679 of FIG. 16 may correspond to steps S1576, S1577, S1578, and S1579 of FIG. 15. Duplicate explanations are omitted.
[0268] If the distance difference between the candidate location and the power-supplyable location is less than the threshold distance (S1675-Y), the electronic device (100) can filter out candidate locations within the threshold distance from the power-supplyable location among the candidate locations (S1676).
[0269] The electronic device (100) can identify a candidate projection surface in the candidate area (S1677). The electronic device (100) can obtain the candidate projection surface location (S1678).
[0270] The electronic device (100) can determine the position closest to the candidate projection surface position among the candidate positions filtered in step S1676 as the target position.
[0271] When there are multiple candidate locations, power supply locations, and candidate projection surface locations, the target location can be determined using each location or the average distance difference between each location. A description of this is provided in FIGS. 14 and 15.
[0272] FIG. 17 is a drawing for explaining a screen that provides a target location according to one embodiment.
[0273] Referring to FIG. 17, the electronic device (100) can provide a screen (1700) related to a target location (e.g., a charging location). The electronic device (100) can provide the screen (1700) through a display.
[0274] The screen (1700) may include at least one of a UI (1710) including text indicating that a target location (e.g., a charging location) is recommended, and a UI (1720) including map data indicating the target location (e.g., a charging location).
[0275] The UI (1720) may include map data indicating at least one of the current location or target location (e.g., charging location) of the electronic device (100).
[0276] FIG. 18 is a drawing for explaining a screen for selecting a target location according to one embodiment.
[0277] Referring to FIG. 18, the electronic device (100) may provide a screen (1800) that guides selection of a target location (e.g., a charging location).
[0278] The screen (1800) may include at least one of a UI (1810) indicating that multiple target locations (e.g., charging locations) have been searched, a UI (1820) guiding selection of a target location (e.g., charging location), and a UI (1830) including map data.
[0279] The UI (1830) may include map data indicating the current location of the electronic device (100) and at least one of a plurality of target locations (e.g., a charging location).
[0280] Assuming that there is one target location (e.g., a charging location) that is ultimately determined, it can be described as determining one target location (e.g., a charging location) among a plurality of candidate locations.
[0281] FIG. 19 is a drawing for explaining a screen related to setting a target location, according to one embodiment.
[0282] Referring to FIG. 19, the electronic device (100) may provide a screen (1900) related to a function for setting a target location (e.g., a charging location). At least one constraint information may be used in determining the target location (e.g., a charging location).
[0283] Limitation information may include various types of information used to determine a target location (e.g., a charging location). Limitation information may be described as threshold information. The limit information may include at least one of the threshold area described in FIG. 8, the threshold ratio described in FIG. 9, the threshold time described in FIG. 13, and the threshold distance described in FIG. 14. A specific drawing is merely an example and is not intended to limit the description of the limit information. Limitation information may also be used in other drawings.
[0284] The electronic device (100) can determine a target location (e.g., a charging location) using the restriction information. The electronic device (100) can change the restriction information based on a preset event.
[0285] For example, the electronic device (100) can change the limit information based on user input. For example, the electronic device (100) can change the threshold distance from 0.3 m to 0.45 m.
[0286] For example, the electronic device (100) may change the restriction information based on a preset ratio. For example, the electronic device (100) may change the threshold distance to a value (0.45 m) that is increased by a threshold ratio (50%) of 0.3 m. The preset ratio may be a ratio intended to further identify a target location (e.g., a charging location) by changing the restriction information. The electronic device (100) may relax the restriction based on the preset ratio.
[0287] The electronic device (100) may not be able to identify a target location (e.g., a charging location). If an event occurs in which a target location (e.g., a charging location) is not identified, the electronic device (100) may provide a screen (1900).
[0288] The screen (1900) may include at least one of a UI (1910) indicating that a target location (e.g., a charging location) is not identified, a UI (1920) guiding changes to restrictions, a UI (1930) explaining the restrictions, a UI (1940) indicating specific settings for the restrictions, and a UI (1950) including map data indicating the restrictions.
[0289] FIG. 20 is a diagram for explaining an operation of determining a target location by considering a user's speech history according to one embodiment.
[0290] The embodiment (2010) of FIG. 20 can acquire the location of a user's speech in a space where an electronic device (100) is located. The electronic device (100) can receive the user's voice. When the user's voice is received, the electronic device (100) can identify the location where the user's voice was spoken. The electronic device (100) can include multiple microphones. The electronic device (100) can identify the location where the user's voice was spoken based on the same user's voice received from multiple microphones.
[0291] The user voice may represent a command to call the electronic device (100). The electronic device (100) may not store the utterance location for all user voices. The electronic device (100) may identify the utterance location only for user voices related to controlling the electronic device (100).
[0292] When receiving a user voice including at least one of a wake-up word related to voice recognition of the electronic device (100) and a control command for performing a control operation of the electronic device (100), the electronic device (100) can identify the location where the received user voice was spoken. The electronic device (100) can accumulate and store the speaking locations. The electronic device (100) can map and store the speaking locations in map data.
[0293] The embodiment (2020) of FIG. 20 may indicate changing the target location based on the utterance location associated with the user's voice. The initially determined target location may be a first target location (2021). After the first target location is determined, the electronic device (100) may accumulate the utterance locations of the user's voice for a threshold period and store them in map data. After the threshold period has elapsed, the electronic device (100) may change the target location from the first target location (2021) to the second target location (2022) based on the utterance location.
[0294] FIG. 21 is a diagram for explaining an operation of outputting a test audio signal by taking into account the utterance location according to one embodiment.
[0295] Steps S2140, S2150, S2160, and S2170 of FIG. 21 may correspond to steps S740, S750, S760, and S770 of FIG. 7. Duplicate explanations are omitted.
[0296] The electronic device (100) can identify the utterance location of the user command (S2125). When a user command is received, the electronic device (100) can identify the utterance location where the user command was uttered.
[0297] The electronic device (100) can determine the ignition location as a representative location (S2130). The electronic device (100) can output a test audio signal based on the representative location (S2140). The electronic device (100) can perform steps S2150, S2160, and S2170.
[0298] FIG. 22 is a drawing for explaining an operation of determining a target position by considering an ignition position according to one embodiment.
[0299] Steps S2272, S2273, S2274, and S2275 of FIG. 22 may correspond to steps S1472, S1473, S1474, and S1475 of FIG. 14. Duplicate explanations are omitted.
[0300] If the distance difference between the candidate location and the power-supplyable location is less than the threshold distance (S2275-Y), the electronic device (100) can filter out candidate locations within the threshold distance from the power-supplyable location among the candidate locations (S2276).
[0301] The electronic device (100) can identify the firing location corresponding to the user command (S2277). The electronic device (100) can determine the location closest to the firing location among the candidate locations filtered in step S2276 as the target location.
[0302] There may be multiple firing locations. For example, the electronic device (100) may identify an average firing location among multiple firing locations and determine a target location based on the identified average firing location. The electronic device (100) may determine the candidate location closest to the average firing location among the filtered candidate locations as the target location (S2279).
[0303] FIG. 23 is a drawing for explaining a control method of an electronic device (100) according to one embodiment.
[0304] Referring to FIG. 23, a control method of an electronic device includes a step of obtaining map data related to a space where the electronic device is located (S2305), a step of identifying a candidate area having a threshold area or more in the map data (S2310), a step of determining a representative position representing the candidate area (S2315), a step of outputting a test audio signal stored in the electronic device based on the representative position (S2320), a step of obtaining a recorded audio signal including the test audio signal (S2325), a step of obtaining a reverberation time of the test audio signal based on the recorded audio signal (S2330), a step of determining a position having the largest reverberation time in the candidate area as a target position (S2335), and a step of providing the target position.
[0305] The step of identifying a candidate area (S2310) divides the map data into multiple areas, determines an area having a threshold area or larger among the multiple areas as a candidate area, and the step of determining a target location (S2335) can determine a target location in the candidate area.
[0306] The step of identifying a candidate region (S2310) may include obtaining a perimeter length of a first region having a threshold area or greater, obtaining a length corresponding to an open space in the first region, dividing the length corresponding to the open space by the perimeter length to obtain a ratio of the open space, and determining the first region as a candidate region if the ratio of the open space is less than the threshold ratio.
[0307] The step of obtaining a reverberation time (S2330) may include obtaining a first volume of the test audio signal at a first point in time based on a recorded audio signal after outputting a test audio signal, obtaining a second volume by multiplying the first volume by a preset reverberation ratio, obtaining a second point in time at which the volume of the test audio signal becomes the second volume, and obtaining the reverberation time based on a difference between the first point in time and the second point in time.
[0308] The step of determining a target position (S2335) may determine a position in a candidate area where the reverberation time is greater than or equal to a threshold time as a candidate position, and determine the target position based on the candidate position and additional information.
[0309] The additional information includes a power supply location for connecting power to the electronic device, and the step (S2335) of determining the target location may obtain a distance difference between the candidate location and the power supply location, and determine a candidate location that is closest to the power supply location among a plurality of candidate locations having a distance difference less than a threshold distance as the target location.
[0310] The additional information includes a candidate projection surface position related to the output of the projection image, and the step of determining the target position (S2335) can determine the candidate position closest to the candidate projection surface position as the target position.
[0311] The recorded audio signal is a first recorded audio signal, the target location is the first target location, and the control method further includes the steps of identifying an utterance location of a user command during a threshold period, outputting a test audio signal based on the utterance location, obtaining a second recorded audio signal including the test audio signal, obtaining a reverberation time of the test audio signal based on the second recorded audio signal, and determining a location in the candidate area where the reverberation time is the largest as the second target location, wherein the second target location may be different from the first target location.
[0312] The control method may further include the step of displaying a UI including map data indicating a target location.
[0313] The control method is to move to the target location when a preset event occurs.
[0314] The methods according to the various embodiments of the present disclosure described above can be implemented in the form of applications that can be installed on existing electronic devices.
[0315] The methods according to the various embodiments of the present disclosure described above can be implemented only with a software upgrade or a hardware upgrade for an existing electronic device.
[0316] The various embodiments of the present disclosure described above may also be performed through an embedded server provided in an electronic device, or an external server of at least one of the electronic device and the display device.
[0317] According to an example embodiment of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device may include an electronic device according to the disclosed embodiments, which is a device that can call instructions stored in the storage medium and operate according to the called instructions. When the instructions are executed by a processor, the processor may directly or under the control of the processor use other components to perform a function corresponding to the instructions. The instructions may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain signals and is tangible, but does not distinguish between whether data is stored semi-permanently or temporarily in the storage medium.
[0318] According to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.
[0319] Each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the respective components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.
[0320] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea of the present disclosure.
Claims
1. In electronic devices, speaker; Mike; and Identifying a candidate area having a critical area or greater in map data related to the space where the electronic device is located; Move to a representative position in the above candidate area and output an audio signal through the speaker, Obtaining the reverberation time of the audio signal based on recorded audio signals corresponding to the audio signal at a plurality of locations including the representative location within the candidate area acquired through the microphone, An electronic device comprising at least one processor for identifying a location at said plurality of locations with the longest reverberation time as a target location.
2. In paragraph 1, At least one processor of the above, In the above map data, an area having a critical area or greater among multiple areas is identified as a candidate area, An electronic device for identifying a target location in the above candidate area.
3. In paragraph 2, At least one processor of the above, Obtain the perimeter of the first region having a critical area or greater, Obtain the length corresponding to the open space in the first region above, An electronic device that identifies the first area as a candidate area if the ratio of the open space is less than a critical ratio when the length corresponding to the open space is divided by the perimeter length.
4. In paragraph 1, At least one processor of the above, After outputting the above audio signal, a first volume of the audio signal is obtained at a first point in time based on the recorded audio signal, Obtain a second volume by multiplying the preset reverberation ratio by the first volume, The above audio signal acquires a second point in time at which the second volume is achieved, An electronic device that obtains the reverberation time based on the difference between the first time point and the second time point.
5. In paragraph 4, At least one processor of the above, Identifying a location where the reverberation time is greater than a threshold time among the above multiple locations as a candidate location, An electronic device that identifies the target location based on the candidate location and additional information.
6. In paragraph 5, The above additional information is: Including a power supply location for connecting power to said electronic device; At least one processor of the above, An electronic device that identifies, as the target location, a candidate location closest to the power-supplyable location among a plurality of candidate locations where the difference in distance between the candidate location and the power-supplyable location is less than a threshold distance.
7. In paragraph 5, The above additional information is: Contains candidate projection surface locations associated with the output of the projection image, At least one processor of the above, An electronic device that identifies the candidate location closest to the candidate projection surface location as the target location.
8. In paragraph 1, The above recorded audio signal is a first recorded audio signal, The above target position is the first target position, At least one processor of the above, Identify the location of the user command's utterance during the critical period, Through the above speaker, the audio signal is output based on the speaking location, Through the microphone, a second recording audio signal including the audio signal is obtained, Obtaining the reverberation time of the audio signal based on the second recorded audio signal, Identifying the location with the longest reverberation time among the above multiple locations as the second target location, An electronic device wherein the second target position is different from the first target position.
9. In paragraph 1, display; including more; At least one processor of the above, An electronic device that controls the display to display a UI including the map data indicating the target location.
10. In paragraph 1, At least one processor of the above, An electronic device that moves to the target location when a preset event occurs.
11. In a method for controlling an electronic device, A step of identifying a candidate area having a threshold area or greater in map data related to a space where the electronic device is located; A step of moving to a representative position of the above candidate area and outputting an audio signal; A step of obtaining recorded audio signals corresponding to the audio signal at a plurality of locations within the candidate area; A step of obtaining the reverberation time of the audio signal at the plurality of locations including the representative location within the candidate area based on the recorded audio signal; and A control method, comprising: a step of identifying a location with the longest reverberation time among the plurality of locations as a target location.
12. In paragraph 11, The step of identifying the above candidate areas is: In the above map data, an area having a critical area or greater among multiple areas is identified as a candidate area, The step of identifying the above target location is: A control method for identifying a target location in the above candidate area.
13. In paragraph 12, The step of identifying the above candidate areas is: Obtain the perimeter of the first region having a critical area or greater, Obtain the length corresponding to the open space in the first region above, A control method for identifying the first area as a candidate area if the ratio of the open space is less than a critical ratio when the length corresponding to the open space is divided by the perimeter length.
14. In paragraph 11, The step of obtaining the above reverberation time is: After outputting the above audio signal, a first volume of the audio signal is obtained at a first point in time based on the recorded audio signal, Obtain a second volume by multiplying the preset reverberation ratio by the first volume, The above audio signal acquires a second point in time at which the second volume is achieved, A control method for obtaining the reverberation time based on the difference between the first point in time and the second point in time.
15. In paragraph 14, The step of identifying the above target location is: Identifying a location where the reverberation time is greater than a threshold time among the above multiple locations as a candidate location, A control method for identifying the target location based on the candidate location and additional information.
Citation Information
Patent Citations
Artificial intelligence system based on speech recognition
CN113299292A
System and method of estimating voice performance for recognizing remote voice
KR1020080111290A
Cosmetic composition for skin whitening comprising hemp seed extract or fraction thereof
KR102437002B1
Multi-directional thermal transmittance measuring device
KR102606602B1
KR20230164837A