Display device

Abstract

A display device according to an embodiment of the present invention may include: a microphone; the controller is used for receiving a microphone starting command for controlling the microphone to be started from the remote control device; and a display that displays music information retrieved based on the audio recorded by the microphone from a time point at which the microphone turn-on command is received.

Description

TECHNICAL FIELD

[0001] The present invention relates to a display apparatus for retrieving music contained in content. BACKGROUND

[0002] An electronic apparatus including a display apparatus can provide various content. According to the diversification of content, the genre of music contained in content can also diversify.

[0003] For example, the audio of content can diversify according to the type of content. In the case where the type of content is a music-related type, the audio of content can contain only music. On the other hand, in the case where the type of content is not a music-related type, music and dialogue sound can be contained together. In the case where the type of content is not a music-related type, the music contained in content can be background music.

[0004] In addition, content can include an original song or a cover song of an original song by another person who is not the original singer, etc.

[0005] The demand for retrieving music of content is increasing. The existing music retrieval apparatus can record audio and retrieve music. As an example of the music retrieval apparatus, there can be a display apparatus that provides a music retrieval function.

[0006] However, the existing display apparatus has a problem in that the process for music retrieval is complicated because an additional step of activating a microphone is required in order to record audio. Thus, there is a problem in that if recording of audio is started in order to retrieve music, the music to be retrieved has already ended. In addition, there is a problem in that the user needs to wait during the time of recording audio.

[0007] In addition, the existing display apparatus has a problem in that the music recognition rate significantly decreases in the case where noise is contained in the audio recording process in which audio recording is performed once.

[0008] Also, the existing display apparatus has a problem in that it is difficult to provide various music information including an original song and a cover song because one music retrieval engine is used. SUMMARY

[0009] PROBLEMS TO BE SOLVED BY THE INVENTION

[0010] The present invention aims to advance the point in time at which a display apparatus starts audio recording for music retrieval.

[0011] The present invention aims to enable music retrieval using other recorded audio in the case where the display apparatus does not obtain a music retrieval result using a part of the recorded audio.

[0012] The present application aims to subdivide the process of the display device performing music search according to music contained in content.

[0013] Technical solutions to solve the problems

[0014] The display device of the embodiment of the present application can include a microphone, a controller receiving a microphone-on command for controlling the microphone to be turned on from a remote control device, and a display displaying music information searched based on audio recorded by the microphone from a time point when the microphone-on command is received.

[0015] The display device of the embodiment of the present application can display music information searched based on audio recorded by the microphone from a time point when a content recommendation command for executing a content recommendation menu is received.

[0016] The display device of the embodiment of the present application can display music information searched based on audio recorded by the microphone from a time point when a start-up word for starting up the display device is received.

[0017] The display device of the embodiment of the present application can start multiple times of recording from a time point when the microphone-on command is received to generate multiple audio files, and can display music information searched based on at least one of the multiple audio files.

[0018] The display device of the embodiment of the present application can start recording at a preset interval from a time point when the microphone-on command is received to generate multiple audio files, and can display music information searched based on at least one of the multiple audio files.

[0019] The display device of the embodiment of the present application can display music information based on an audio file selected according to at least one of a ratio of a mute interval of each of the multiple audio files and an SNR (Signal to Noise Ratio) level.

[0020] The display device of the embodiment of the present application can display music information based on an audio file whose ratio of a mute interval in the multiple audio files is less than a preset reference ratio.

[0021] The display device of the embodiment of the present application can display music information based on an audio file whose SNR level in the multiple audio files is greater than or equal to a preset reference level.

[0022] The display device of the embodiment of the present application can acquire an order of an audio file for music search based on at least one of a ratio of a mute interval of each of the selected audio files and an SNR level.

[0023] The display apparatus according to an embodiment of the present application can acquire an audio file having the smallest ratio of a mute interval among the filtered audio files as an audio file used as a first priority in music search, and acquire an audio file having a ratio of a mute interval next to the first audio file as an audio file used as a second priority in music search.

[0024] The display apparatus according to an embodiment of the present application can acquire an audio file having the largest SNR level among the filtered audio files as an audio file used as a first priority in music search, and acquire an audio file having an SNR level next to the first audio file as an audio file used as a second priority in music search.

[0025] The display apparatus according to an embodiment of the present application can acquire whether to remove a signal other than music included in an audio based on a type of content from an external server.

[0026] The display apparatus according to an embodiment of the present application can display music information searched based on an audio in which a signal other than a music signal is removed, in a case where the type of content is not a music-related type.

[0027] The display apparatus according to an embodiment of the present application can perform music search corresponding to an audio recorded using the microphone in at least one music search engine among a plurality of music search engines including an original song search engine and a cover song search engine.

[0028] The display apparatus according to an embodiment of the present application can perform music search in a cover song search engine in a case where a music search result is not acquired from a song search engine.

[0029] Effects of Invention

[0030] The display apparatus according to an embodiment of the present application can shorten a time between a point in time at which a music search command is received and a point in time at which a music search result is displayed.

[0031] The display apparatus according to an embodiment of the present application sequentially performs music search using a plurality of audio files, and thus can minimize a possibility that a search result is not acquired.

[0032] The display apparatus according to an embodiment of the present application can reduce a calculation amount based on music search or shorten a time until a point in time at which a music search result is displayed, according to music included in content. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 A configuration of a display apparatus according to an embodiment of the present application is illustrated in a block diagram.

[0034] Figure 2 is a block diagram of a remote control apparatus according to an embodiment of the present application.

[0035] Figure 3 An actual configuration example of the remote control device of an embodiment of the present application is shown.

[0036] Figure 4 An example of using the remote control device according to an embodiment of the present application is shown.

[0037] Figure 5 is a block diagram for explaining a mobile terminal related to the present application.

[0038] Figure 6 is a diagram for explaining a time point at which an existing display device starts audio recording for music search.

[0039] Figure 7 is a flowchart for explaining a method of the display device of an embodiment of the present application searching for music.

[0040] Figure 8 is a diagram for explaining a case where the display device of an embodiment of the present application receives a music search command after completing audio recording.

[0041] Figure 9 is a diagram for explaining a case where the display device of an embodiment of the present application receives a music search command before completing audio recording.

[0042] Figure 10 is a diagram illustrating a content recommendation menu displayed by the display device of an embodiment of the present application according to a content recommendation command.

[0043] Figure 11 is a diagram illustrating a menu displayed by a mobile terminal connected to the display device of an embodiment of the present application in order to control the display device.

[0044] Figure 12 is a diagram illustrating a form of a menu displayed by a mobile terminal connected to the display device of an embodiment of the present application for controlling the display device.

[0045] Figure 13 is a diagram illustrating a screen displayed by the display device of an embodiment of the present application as the display device receives a start-up word.

[0046] Figure 14 is a diagram illustrating a screen in which the display device of an embodiment of the present application displays a menu including a music search item.

[0047] Figure 15 is a diagram illustrating a method of the display device of an embodiment of the present application starting multiple recordings.

[0048] Figure 16 is a flowchart for explaining a method of the display device of an embodiment of the present application searching for music information using a plurality of files recorded.

[0049] Figure 17 FIG. 1 is a diagram for explaining a method of a display apparatus of an embodiment of the present application for selecting audio files for music search based on a ratio of respective silent sections of the audio files.

[0050] Figure 18 FIG. 2 is a diagram for explaining a method of a display apparatus of an embodiment of the present application for selecting audio files for music search based on respective SNR (Signal to Noise Ratio) levels of the audio files. DETAILED DESCRIPTION

[0051] Hereinafter, embodiments related to the present application will be described in greater detail with reference to the accompanying drawings. The suffixes "module" and "part" used for components in the following description are merely intended for facilitating a description of the specification, and do not have distinct meanings or roles in some cases.

[0052] The display apparatus of the embodiment of the present application is, for example, a smart display apparatus to which a computer support function is additionally attached on a broadcast receiving function, and has an Internet function or the like additionally attached while fully performing the broadcast receiving function, so that it can have a more convenient interface such as a handwriting type input device or a touch screen or a spatial remote controller. Further, by being connected to the Internet and a computer with the support of a wired or wireless Internet function, it is also capable of performing functions such as an e-mail or web browsing or banking or gaming. In order to implement such various functions, a standardized general OS can be used.

[0053] Therefore, in the display apparatus described in the present application, for example, various application programs can be freely added or deleted on a general OS kernel, so that various functions friendly to a user can be performed. More specifically, the display apparatus can be, for example, a network TV, an HBBTV, a smart TV, an LED TV, an OLED TV, or the like, and can also be applied to a smart phone according to circumstances.

[0054] Figure 1 The configuration of the display apparatus of an embodiment of the present application is illustrated in a block diagram.

[0055] Reference Figure 1 The display apparatus 100 can include a broadcast receiving part 130, an external device interface 135, a storage 140, a user input interface 150, a controller 170, a wireless communication interface 173, a display 180, a speaker 185, and a power supply circuit 190.

[0056] The broadcast receiving part 130 can include a tuner 131, a demodulator 132, and a network interface 133.

[0057] The tuner 131 can select a specific broadcast channel according to a channel selection instruction. The tuner 131 can receive a broadcast signal of the selected specific broadcast channel.

[0058] The demodulator 132 can separate the received broadcast signal into a video signal, an audio signal, and a data signal related to a broadcast program, and can restore the separated video signal, audio signal, and data signal into a form that can be output.

[0059] The external device interface 135 can receive an application or an application directory in a neighboring external device and deliver to the controller 170 or the memory 140.

[0060] The external device interface 135 can provide a connection path between the display device 100 and an external device. The external device interface 135 can receive one or more of video, audio, etc. output from an external device connected to the display device 100 in a wireless or wired manner and deliver to the controller 170. The external device interface 135 can include a plurality of external input terminals. The plurality of external input terminals can include an RGB terminal, one or more HDMI (High Definition Multimedia Interface) terminals, a Component terminal.

[0061] The video signal of the external device input through the external device interface 135 can be output through the display 180. The voice signal of the external device input through the external device interface 135 can be output through the speaker 185.

[0062] The external device capable of being connected with the external device interface 135 can be, for example, any one of a set-top box, a Blu-ray player, a DVD player, a game console, a sound bar, a smartphone, a PC, a USB memory, a home theater, but this is only an example.

[0063] The network interface 133 can provide an interface for connecting the display device 100 to a wired / wireless network including the Internet. The network interface 133 can perform transmission or reception of data with other users or other electronic devices through the accessed network or other networks linked with the accessed network.

[0064] In addition, a part of the content data stored in the display device 100 can be transmitted to a user or a selected electronic device selected from among other users or other electronic devices pre-registered to the display device 100.

[0065] The network interface 133 can connect a prescribed web page through the connected network or other networks linked with the connected network. That is, by connecting the prescribed web page through the network, data can be transmitted or received with a corresponding server.

[0066] In addition, the network interface 133 can receive content or data provided by a content provider or a network operator. That is, the network interface 133 can receive content such as a movie, an advertisement, a game, a VOD, a broadcast signal, etc. provided by a content provider or a network provider and information related thereto through a network.

[0067] In addition, the network interface 133 can receive update information and an update file of firmware provided by a network operator, and can transmit data to the Internet or a content provider or a network operator.

[0068] The network interface 133 can select and receive an application required in an application program that is open to the public through a network.

[0069] The storage 140 can store a program for processing and controlling various signals within the controller 170, and can store an image or voice or data signal processed by a signal.

[0070] In addition, the storage 140 can further perform a function for temporarily storing an image or voice or data signal input from the external device interface 135 or the network interface 133, and can store information related to a predetermined image through a channel memory function.

[0071] The storage 140 can store an application program or an application program directory input from the external device interface 135 or the network interface 133.

[0072] The display device 100 can play a content file (a video file, a still image file, a music file, a text file, an application file, etc.) stored in the storage 140 to provide to a user.

[0073] The user input interface 150 can deliver a signal input by a user to the controller 170 or deliver a signal from the controller 170 to the user. For example, the user input interface 150 can receive a power on / off, a channel selection, a screen setting, etc. control signal from the remote control device 200 based on various communication methods such as a Bluetooth, a UWB (Ultra Wideband), a ZigBee, an RF (Radio Frequency) communication method, or an IR (Infrared Ray) communication method, and process, or can process a control signal from the controller 170 to be transmitted to the remote control device 200.

[0074] In addition, the user input interface 150 can deliver a control signal input from a power key, a channel key, a volume key, a setting key, etc. local key (not illustrated) to the controller 170.

[0075] The video signal processed by the controller 170 can be input to the display 180 and displayed as a video corresponding to the corresponding video signal. In addition, the video signal processed by the controller 170 can be input to an external output device through the external device interface 135.

[0076] The voice signal processed by the controller 170 can be audio output to the speaker 185. In addition, the voice signal processed by the controller 170 can be input to an external output device through the external device interface 135.

[0077] In addition, the controller 170 can control the overall operation within the display device 100.

[0078] In addition, the controller 170 can control the display device 100 using a user instruction input through the user input interface 150 or an internal program, and can be connected to a network to allow a user to download a desired application program or an application program directory into the display device 100.

[0079] The controller 170 enables channel information or the like selected by a user to be output together with a processed video or voice signal through the display 180 or the speaker 185.

[0080] In addition, the controller 170 enables a video signal or a voice signal from an external device, such as a camera or a camcorder, input through the external device interface 135 to be output through the display 180 or the speaker 185 according to an external device video play instruction received through the user input interface 150.

[0081] On the other hand, the controller 170 can control the display 180 to display a video, for example, can control the display 180 to display a broadcast video input through the tuner 131, or an external input video input through the external device interface 135, or a video input through the network interface unit, or a video stored in the storage 140. In this case, the video displayed on the display 180 can be a still video or a video, and can be a 2D video or a 3D video.

[0082] In addition, the controller 170 can control to play a content stored in the display device 100, or a received broadcast content, or an external input content externally input, which can be a broadcast video, an external input video, an audio file, a still video, a connected web page screen, and a text file, etc. in various forms.

[0083] The wireless communication interface 173 can perform communication with an external device through wired or wireless communication. The wireless communication interface 173 can perform short range communication with an external device. To this end, the wireless communication interface 173 can utilize Bluetooth (Bluetooth®), Zigbee (Zigbee®), infrared data association (IrDA), wireless fidelity (Wi-Fi), wireless broadband (WiBro), world interoperability for microwave access (WiMAX), or global system for mobile communications (GSM), code division multiple access (CDMA), digital advanced mobile phone system (D-AMPS), or general packet radio service (GPRS) communication.TM ), RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, Wireless Universal Serial Bus (WirelessUSB) technology. Such a wireless communication interface 173 can support wireless communication between the display apparatus 100 and a wireless communication system, or between the display apparatus 100 and other display apparatuses 100, or between the display apparatus 100 and a network in which the display apparatus 100 (or an external server) is located, through a wireless area network. The wireless area network can be a wireless personal area network.

[0084] Here, the other display apparatus 100 can be a wearable device (e.g., a smart watch, a smart glass, an HMD (head mounted display), a smart phone, etc.) that can exchange data with the display apparatus 100 of the present application (or be linkable). The wireless communication interface 173 can detect (or identify) a wearable device that is communicable around the display apparatus 100.

[0085] Further, in the case where the detected wearable device is a device that is verified to communicate with the display apparatus 100 of the present application, the controller 170 can transmit at least a part of data processed in the display apparatus 100 to the wearable device through the wireless communication interface 173. Accordingly, a user of the wearable device can utilize the data processed in the display apparatus 100 through the wearable device.

[0086] The display 180 can generate a driving signal by converting an image signal, a data signal, an OSD signal processed in the controller 170, or an image signal, a data signal, etc. received from the external device interface 135 into R, G, B signals, respectively.

[0087] On the other hand, Figure 1The display device 100 shown is only one embodiment of the present invention. Some of the constituent elements shown may be integrated, added, or omitted according to the specifications of the actual implemented display device 100.

[0088] That is, two or more constituent elements can be combined into one constituent element, or one constituent element can be subdivided into two or more constituent elements, as needed. Furthermore, the functions performed in each box are intended to illustrate embodiments of the invention, and their specific actions or devices do not limit the scope of the invention.

[0089] According to another embodiment of the present invention, with Figure 1 Unlike the previous one, the display device 100 may also not have a tuner 131 and a demodulator 132, and may receive and play images through a network interface 133 or an external device interface 135.

[0090] For example, the display device 100 can be divided into an image processing device, such as a set-top box for receiving content based on broadcast signals or various network services, and a content playback device for playing content input from the image processing device.

[0091] In this case, the operation method of the display device of the embodiments of the present invention described below can be derived not only from the reference Figure 1 The operation can be performed by the display device 100 described above, or by any one of the image processing devices such as the set-top box and the content playback device having a display 180 and an audio output unit 185.

[0092] Next, refer to Figures 2-3 The following describes a remote control device according to an embodiment of the present invention.

[0093] Figure 2 This is a block diagram of a remote control device according to an embodiment of the present invention. Figure 3 An actual configuration example of a remote control device 200 according to an embodiment of the present invention is shown.

[0094] First, refer to Figure 2 The remote control device 200 may include a fingerprint reader 210, a wireless communication circuit 220, a user input interface 230, a sensor 240, an output interface 250, a power supply circuit 260, a memory 270, a controller 280, and a microphone 290.

[0095] Reference Figure 2 The wireless communication circuit 220 transmits and receives signals from any of the display devices in the aforementioned embodiments of the present invention.

[0096] The remote control apparatus 200 can have an RF circuit 221 capable of transceiving a signal with the display apparatus 100 according to an RF communication standard, and can have an IR circuit 223 capable of transceiving a signal with the display apparatus 100 according to an IR communication standard. In addition, the remote control apparatus 200 can have a Bluetooth circuit 225 capable of transceiving a signal with the display apparatus 100 according to a Bluetooth communication standard. In addition, the remote control apparatus 200 can have an NFC (Near Field Communication) circuit 227 capable of transceiving a signal with the display apparatus 100 according to an NFC communication standard, and can have a WLAN (Wireless LAN) circuit 229 capable of transceiving a signal with the display apparatus 100 according to a WLAN communication standard.

[0097] In addition, the remote control apparatus 200 transmits a signal including information related to movement of the remote control apparatus 200, etc. to the display apparatus 100 through the wireless communication circuit 220.

[0098] On the other hand, the remote control apparatus 200 can receive a signal transmitted by the display apparatus 100 through the RF circuit 221, and as needed, can transmit an instruction regarding power on / off, channel change, volume change, etc. to the display apparatus 100 through the IR circuit 223.

[0099] The user input interface 230 can be constituted by a keyboard or a key or a touchpad or a touch screen, etc. A user can operate the user input interface 230 to input an instruction related to the display apparatus 100 to the remote control apparatus 200. In the case where the user input interface 230 has a hard key, a user can input an instruction related to the display apparatus 100 to the remote control apparatus 200 through a press action of the hard key. In this regard, reference is made to Figure 3 for an explanation.

[0100] Referring to Figure 3 , the remote control apparatus 200 can include a plurality of keys. The plurality of keys can include a fingerprint recognition key 212, a power key 231, a home key 232, a live broadcast key 233, an external input key 234, a volume adjustment key 235, a voice recognition key 236, a channel change key 237, a confirmation key 238, and a return key 239.

[0101] The fingerprint recognition key 212 can be a key for recognizing a fingerprint of a user. As an embodiment, the fingerprint recognition key 212 can perform a press action, and thus can receive a press action and a fingerprint recognition action.

[0102] The power key 231 can be a key for turning on / off a power of the display apparatus 100.

[0103] The home key 232 can be a key for moving to a home screen of the display apparatus 100.

[0104] The live key 233 is a key for displaying a real-time broadcast program.

[0105] The external input key 234 can be a key for receiving an external input connected to the display apparatus 100.

[0106] The volume adjustment key 235 can be a key for adjusting a volume level of an output of the display apparatus 100.

[0107] The voice recognition key 236 can be a key for receiving a user’s voice and recognizing the received voice.

[0108] The channel change key 237 can be a key for receiving a broadcast signal of a specific broadcast channel.

[0109] The confirmation key 238 can be a key for selecting a specific function, and the return key 239 can be a key for returning to a previous screen.

[0110] Further explanation Figure 2 .

[0111] In a case where the user input interface 230 is provided with a touch screen, a user can input an instruction related to the display apparatus 100 to the remote control apparatus 200 by touching a soft key of the touch screen. In addition, the user input interface 230 can have various input devices, such as a scroll key or a jog key, which a user can operate, and the present embodiment does not limit the scope of the present application.

[0112] The sensor 240 can have a gyro sensor 241 or an acceleration sensor 243, and the gyro sensor 241 can sense information related to a movement of the remote control apparatus 200.

[0113] For example, the gyro sensor 241 can sense information related to a motion of the remote control apparatus 200 with x, y, and z axes as a reference, and the acceleration sensor 243 can sense information related to a moving speed or the like of the remote control apparatus 200. On the other hand, the remote control apparatus 200 can further have a distance measurement sensor, and thus can sense a distance between the remote control apparatus 200 and the display 180 of the display apparatus 100.

[0114] The output interface 250 can output an image signal or a voice signal corresponding to an operation of the user input interface 230 or a signal transmitted from the display apparatus 100.

[0115] A user can recognize, through the output interface 250, whether the user input interface 230 is operated or the display apparatus 100 is controlled.

[0116] For example, the output interface 250 can have an LED 251 or a vibrator 253 or a speaker 255 or a display 257, and if the operation user input interface 230 or transmits / receives a signal with the display apparatus 100 through the wireless communication part 225, the LED 251 is lighted, the vibrator 253 generates vibration, the speaker 255 outputs sound, and the display 257 outputs an image.

[0117] In addition, the power supply circuit 260 supplies power to the remote control apparatus 200, and can stop the power supply in case that the remote control apparatus 200 does not act for a prescribed time, thereby reducing power waste.

[0118] The power supply circuit 260 can resume the power supply in case that a prescribed key provided in the remote control apparatus 200 is operated.

[0119] The memory 270 can store various programs, application data, etc. required for the control or the action of the remote control apparatus 200.

[0120] In case that the remote control apparatus 200 transmits / receives a signal with the display apparatus 100 through the RF circuit 221, the remote control apparatus 200 and the display apparatus 100 transmit / receive a signal through a prescribed frequency band.

[0121] The controller 280 of the remote control apparatus 200 can store information about a frequency band, etc. capable of transmitting / receiving a signal wirelessly with the display apparatus 100 paired with the remote control apparatus 200 to the memory 270 and refer to the information.

[0122] The controller 280 controls various matters related to the control of the remote control apparatus 200. The controller 280 can transmit a signal corresponding to the operation of a prescribed key of the user input interface 230 or a signal corresponding to the movement of the remote control apparatus 200 sensed by the sensor 240 to the display apparatus 100 through the wireless communication part 225.

[0123] In addition, the microphone 290 of the remote control apparatus 200 can acquire a voice.

[0124] The microphone 290 can be provided in plural.

[0125] Next, an example of using the remote control apparatus according to an embodiment of the present application will be described. Figure 4

[0126] Figure 4 An example of using the remote control apparatus according to an embodiment of the present application will be described.

[0127] Figure 4 (a) of FIG. 1 illustrates that the cursor 205 corresponding to the remote control apparatus 200 is displayed on the display 180.

[0128] ​The user can move the remote control device 200 up and down, left and right, or rotate it. The cursor 205 displayed on the display 180 of the display device 100 corresponds to the movement of the remote control device 200. As shown, since this remote control device 200 moves the corresponding cursor 205 and displays it as it moves in the 3D space, it can be named as a spatial remote controller.

[0129] Figure 4 (b) of FIG. 1 illustrates that when the user moves the remote control device 200 to the left, the cursor 205 displayed on the display 180 of the display device 100 also moves to the left in correspondence thereto.

[0130] Information about the movement of the remote control device 200 detected by the sensor of the remote control device 200 is transmitted to the display device 100. The display device 100 can calculate the coordinates of the cursor 205 from the information about the movement of the remote control device 200. The display device 100 can display the cursor 205 in correspondence to the calculated coordinates.

[0131] Figure 4 (c) of FIG. 1 illustrates a case where the user moves the remote control device 200 away from the display 180 in a state where a specific button inside the remote control device 200 is pressed. Thereby, a selection area inside the display 180 corresponding to the cursor 205 can be zoomed in and displayed in an enlarged manner.

[0132] In contrast, in a case where the user moves the remote control device 200 close to the display 180, the selection area inside the display 180 corresponding to the cursor 205 can be zoomed out and displayed in a reduced manner.

[0133] On the other hand, it is also possible that the selection area is zoomed out in a case where the remote control device 200 is moved away from the display 180, and the selection area is zoomed in in a case where the remote control device 200 is moved close to the display 180.

[0134] In addition, in a state where a specific button inside the remote control device 200 is pressed, the up and down, left and right movements can be excluded from recognition. That is, in a case where the remote control device 200 is moved close to or away from the display 180, only the front and back movement can be recognized, and the up and down, left and right movements can not be recognized. In a state where the specific button inside the remote control device 200 is not pressed, only the cursor 205 is moved according to the up and down, left and right movement of the remote control device 200.

[0135] On the other hand, the moving speed or moving direction of the cursor 205 can correspond to the moving speed or moving direction of the remote control device 200.

[0136] On the other hand, the cursor in the present specification indicates an object displayed on the display 180 in correspondence with the action of the remote control device 200. Thus, the cursor 205 can be an object of various shapes other than the arrow shape shown in the drawing. For example, it can be a concept including a dot, a pointer, a prompt, a thick outline, etc. Also, the cursor 205 can be displayed in correspondence with not a single point but a plurality of points such as a line, a surface, etc. on the display 180.

[0137] Next, the configuration of a mobile terminal related to the present application will be described with reference to Figure 5

[0138] Figure 5 is a block diagram for explaining a mobile terminal related to the present application.

[0139] The mobile terminal 500 can include a wireless communication part 510, an input part 520, a sensing part 540, an output part 550, an interface part 560, a storage 570, a control part 580, a power supply part 590, etc. Figure 5 The plurality of constituent elements shown are not necessarily essential for implementing a mobile terminal, and the mobile terminal described in the present specification can have more or less constituent elements than those listed above.

[0140] More specifically, the wireless communication part 510 among the plurality of constituent elements can include one or more modules capable of implementing wireless communication between the mobile terminal 500 and a wireless communication system, between the mobile terminal 500 and another mobile terminal 500, or between the mobile terminal 500 and an external server. In addition, the wireless communication part 510 can include one or more modules for connecting the mobile terminal 500 to one or more networks.

[0141] The wireless communication part 510 can include at least one of a broadcast receiving module 511, a mobile communication module 512, a wireless Internet module 513, a close proximity communication module 514, a location information module 515.

[0142] The input part 520 can include a camera 521 or an image input part for inputting an image signal, a microphone 522 or an audio input part for inputting an audio signal, a user input part 523 (e.g., a touch key, a mechanical key, etc.) for receiving information from a user. Voice data or image data collected in the input part 520 can be analyzed and processed as a user's control command.

[0143] ​The sensing unit 540 can include one or more sensors for sensing at least one of information within the mobile terminal, surrounding environment information of the mobile terminal, and user information. For example, the sensing unit 540 can include at least one of a proximity sensor 541, an illumination sensor 542, a touch sensor, an acceleration sensor, a magnetic sensor, a G-sensor, a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor, a finger scan sensor, an ultrasonic sensor, an optical sensor, a microphone, a battery gauge, an environment sensor (for example, a barometer, a hygrometer, a thermometer, a radiation detection sensor, a heat detection sensor, a gas detection sensor, etc.), a chemical sensor (for example, an electronic nose, a medical sensor, a biometric sensor, etc.). In another aspect, the mobile terminal disclosed in the specification can combine information sensed by at least two of the sensors.

[0144] The output unit 550 is for generating an output related to a visual, an auditory, or a tactile, etc., and can include at least one of a display unit 551, a sound output unit 552, a haptic module 553, and an optical output unit 554. The display unit 551 can be formed with a layer structure or integrated with the touch sensor to constitute a touch screen. The touch screen can function as the user input unit 523 for providing an interface between the mobile terminal 500 and a user, as well as an output interface between the mobile terminal 500 and the user.

[0145] The interface unit 560 performs a channeling function between various kinds of external devices connected to the mobile terminal 500. Such an interface unit 560 can include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O (Input / Output) port, a video I / O (Input / Output) port, an earphone port, etc. In the mobile terminal 500, a corresponding control related to the connected external device can be performed by the interface unit 560.

[0146] In addition, the memory 570 stores data supporting various functions of the mobile terminal 500. The memory 570 can store a plurality of application programs (application programs or applications) driven in the mobile terminal 500, data, and instructions for the operation of the mobile terminal 500. At least a part of the application programs can be downloaded from an external server through wireless communication. In addition, at least a part of the application programs can exist on the mobile terminal 500 from the time of manufacture for basic functions (e.g., a call function, a message reception and transmission function, etc.) of the mobile terminal 500. On the other hand, the application programs can be stored in the memory 570, installed on the mobile terminal 500, and driven to perform the operation (or function) of the mobile terminal by the control unit 580.

[0147] The control unit 580 generally controls the overall operation of the mobile terminal 500 in addition to the operation related to the application programs. The control unit 580 processes signals, data, information, etc. input or output through the above-described constituent elements or drives the application programs stored in the memory 570, thereby providing or processing appropriate information or functions for the user.

[0148] In addition, in order to drive the application programs stored in the memory 570, the control unit 580 can control at least a part of the plurality of constituent elements described together. Figure 5 Further, in order to drive the application programs, the control unit 580 can make at least two of the plurality of constituent elements included in the mobile terminal 500 act in combination with each other.

[0149] The power supply unit 590 receives an external power supply, an internal power supply under the control of the control unit 580 and supplies power to each of the constituent elements included in the mobile terminal 500. Such a power supply unit 590 includes a battery, which can be an internal battery or a battery in a replaceable form.

[0150] To implement the actions, controls, or control methods of the mobile terminal of the various embodiments described below, at least some of the various constituent elements can act in cooperation with each other. Also, the actions, controls, or control methods of the mobile terminal can be implemented on the mobile terminal through driving of at least one application program stored in the memory 570.

[0151] In the present specification, the display device 100 can be a music search device that provides a music search function. However, the music search device can not only be the display device 100, but also the mobile terminal 500 described below. That is, the music search device, as a device that displays a result of searching for music according to a method described below, can have various forms, such as the display device 100 or the mobile terminal 500, etc. Figure 5

[0152] Hereinafter, for convenience of explanation, it is assumed that the music search device is the display device 100.

[0153] As an example of a method of searching for music by the display device 100, there is a method of searching for music information using audio recorded by the microphone 151. That is, the display device 100 can search for music having a signal similar to a signal of audio recorded by the microphone 151. Thereby, the display device 100 can provide music information having an audio signal similar to the audio recorded by the microphone 151.

[0154] On the other hand, in order to search for music using recording of audio by the microphone 151, the existing display device 100 goes through one or more steps.

[0155] For example, the existing display device 100 receives a voice command for music search and starts audio recording after going through a step of turning on the microphone 151.

[0156] Alternatively, the existing display device 100 receives a user input selecting a music search function included in an application program installed in the display device 100 and starts audio recording after going through a step of executing the application program.

[0157] Referring to Figure 6 , a point in time at which the existing display device 100 starts audio recording for music search is described.

[0158] Figure 6 is a graph for explaining a point in time at which the existing display device starts audio recording for music search.

[0159] Figure 6 may be a graph representing a waveform of audio that can be recorded by the microphone 151.

[0160] ​The controller 170 can record the audio from a time point t2 at which the music search command is received. The time point t2 at which the music search command is received can be a time point t2 at which the controller 170 starts the audio recording. The music search command can be a command for searching for music.

[0161] The controller 170 can enter a state capable of receiving the music search command before the music search command is received.

[0162] For example, the controller 170 can receive the music search command through the microphone 151. To this end, the controller 170 can enter the state capable of receiving the music search command by receiving a microphone-on command.

[0163] The time point t1 at which the controller 170 enters the state capable of receiving the music search command can be a time point t1 at which the user plans the music search.

[0164] However, between the time point t1 at which the user plans the music search and the time point t2 at which the audio recording is started, the music included in the content can be interrupted or changed to other music.

[0165] Thus, at the time point t2 at which the controller 170 starts the audio recording, the audio including the music to be searched for by the user cannot be recorded. Thus, there is a problem in that the user cannot search for the music to be searched for.

[0166] On the other hand, the controller 170 can record the audio during a preset time d from the time point t2 at which the music search command is received. The preset time d can be set to a minimum recording time capable of obtaining the music search result using the recorded audio, but is not limited thereto.

[0167] The time point t3 at which the audio recording is completed can be after the preset time d from the time point t2 at which the music search command is received.

[0168] If the controller 170 completes the audio recording, music information searched for based on the recorded audio can be displayed.

[0169] Thus, the time point at which the music information searched for by the controller 170 is displayed can be after the time point t3 at which the audio recording is completed.

[0170] Therefore, in the case where the controller 170 records the audio from the time point t2 at which the music search command is received, the time between the time point t1 at which the user plans the music search and the time point at which the searched-for music information is displayed can be longer than the preset time d.

[0171] Thus, the existing display apparatus 100 has a problem in that the preset time d or more is required from the time point t2 at which the music search command is received to the time point at which the searched-for music information is provided.

[0172] Accordingly, the display apparatus 100 of an embodiment of the present application aims to start audio recording for music search at a time point before a time point t2 at which a music search command is received.

[0173] Next, a method of searching for music by the display apparatus 100 of an embodiment of the present application will be described.

[0174] Figure 7 is a flowchart for explaining a method of searching for music by the display apparatus of an embodiment of the present application.

[0175] The controller 170 can receive a microphone-on command to control the microphone 151 to be turned on (S11).

[0176] For example, the controller 170 can receive the microphone-on command to control the microphone 151 to be turned on from the remote control apparatus 200.

[0177] To receive a voice command of a user, the microphone-on command can be a command to turn on a power of the microphone 151. The voice command can include a music search command.

[0178] The controller 170 can receive the music search command through the microphone 151 controlled to be turned on.

[0179] The music search command can be a command to search for music of content. For example, the music search command can be a command to search for music of content displayed through the display 180.

[0180] On the other hand, the step of receiving the microphone-on command by the controller 170 can be a step of entering a state capable of receiving the music search command.

[0181] The state capable of receiving the music search command can be various according to embodiments. For example, the state capable of receiving the music search command can include not only a state in which the microphone 151 is turned on to be capable of receiving the music search command, but also a state of entering a menu supporting the music search command, etc.

[0182] The controller 170 can record audio from a time point at which the microphone-on command is received (S13).

[0183] The controller 170 can record audio from a time point at which the microphone-on command is received. The controller 170 can start audio recording at a time point at which the microphone-on command is received.

[0184] That is, the controller 170 can record audio from before the music search command is received.

[0185] The controller 170 can record the audio for a preset time from a time point at which the audio recording is started.

[0186] The preset time can be set to a minimum recording time capable of acquiring a music search result using the recorded audio, but is not limited thereto. For example, the preset time can be 6 seconds, but is not limited thereto.

[0187] According to an embodiment, the controller 170 can start the audio recording in various cases in which a state capable of receiving a music search command is entered.

[0188] The controller 170 can receive a music search command (S15).

[0189] The controller 170 can receive the music search command through the microphone 151.

[0190] The controller 170 can perform music search based on the recorded audio (S17).

[0191] If the music search command is received, the controller 170 can perform the music search. For example, if the music search command is received, the controller 170 can perform music search corresponding to the audio in a music search engine.

[0192] The music search engine can be a search engine that acquires and provides music information most similar to the audio from a music database.

[0193] For example, the music search engine can acquire and provide music information having a frequency most similar to a frequency of the audio file from a music database after analyzing the frequency of the audio file.

[0194] The music search engine can include an original song search engine and a cover song search engine.

[0195] The original song search engine can be a search engine for acquiring source information most similar to the audio file from an original song database. The original song database can be a database in which an audio file including a sound of an original singer and an instrument sound included in music is stored.

[0196] The cover song search engine can be a search engine for acquiring cover song information most similar to the audio file from a cover song database. The cover song database can be a database in which an audio file including a sound of a person other than an original singer included in music and an instrument sound is stored.

[0197] The controller 170 can perform music search corresponding to the audio recorded with the microphone 151 in at least one of a plurality of music search engines including the original song search engine and the cover song search engine.

[0198] The controller 170 of steps S13 to S17 records the audio, and with respect to a time point at which music search is performed based on the recorded audio, refers to Figures 8-9 The above will be described in detail.

[0199] Figure 8 FIG. 1 is a diagram for explaining a case in which the display apparatus of an embodiment of the present application receives a music search command after completing audio recording.

[0200] Figure 8 may be a graph indicating a waveform of audio that can be recorded through the microphone 151.

[0201] The controller 170 can start audio recording at a time point t1 at which a state in which a music search command can be received is entered. The time point t1 at which the state in which the music search command can be received is entered can be a time point t1 at which the user plans music search.

[0202] The controller 170 can record audio for a preset time d from the time point t1 at which the user plans music search. The preset time d can be six seconds, but is not limited thereto.

[0203] The time point t3 at which the controller 170 completes audio recording can be before the time point t2 at which the music search command is received.

[0204] Thus, if the music search command is received, the controller 170 can directly perform music search based on the recorded audio.

[0205] Figure 9 FIG. 2 is a diagram for explaining a case in which the display apparatus of an embodiment of the present application receives a music search command before completing audio recording.

[0206] Figure 9 may be a graph indicating a waveform of audio that can be recorded through the microphone 151.

[0207] The controller 170 can receive the music search command in the process of recording the audio. The time point t3 at which the controller 170 completes audio recording can be after the time point t2 at which the music search command is received.

[0208] The controller 170 can perform music search based on the audio recorded at the time point t3 at which audio recording is completed.

[0209] The controller 170 starts audio recording at the time point t1 at which the user plans music search, and thus a time between the time point t2 at which the music search command is received and the time point t3 at which audio recording is completed can be shorter than the preset time d.

[0210] Figures 8-9 The display apparatus 100 of the embodiment of FIG. 1 can start audio recording at the time point t1 at which the user plans music search.

[0211] Again, the controller 170 can display the music information retrieved based on the recorded audio (S19). Figure 7 .

[0212] The controller 170 can display the music information retrieved based on the recorded audio.

[0213] The controller 170 can display the music information retrieved based on the recorded audio.

[0214] The retrieved music information can include a title of the retrieved music, a singer, an album name, an album cover, a release date, etc.

[0215] In addition, the controller 170 can display the accuracy of the retrieved music together. The accuracy of the retrieved music can be a degree of coincidence of the retrieved music and the recorded audio.

[0216] Summary Figure 7 The display apparatus 100 according to an embodiment of the present application can record audio for music retrieval from a point in time at which the user plans music retrieval before receiving a music retrieval command.

[0217] Thus, the display apparatus 100 according to an embodiment of the present application can minimize a problem in which music that the user plans to retrieve is interrupted and cannot be recorded before starting audio recording.

[0218] In addition, the display apparatus 100 according to an embodiment of the present application has an advantage of being able to shorten a time until a music retrieval result is displayed.

[0219] The case in which the start of audio recording is advanced to a point in time at which the user wants to retrieve music is not limited to the foregoing case.

[0220] That is, the display apparatus 100 according to an embodiment of the present application can directly start audio recording in a case in which it is determined that a state in which a music retrieval command can be received is satisfied, in addition to a microphone-on command. Hereinafter, various embodiments will be specifically described.

[0221] Relatedly, a description will be made with reference to Figures 10-14 .

[0222] Figure 10 is a diagram illustrating a content recommendation menu displayed by the display apparatus according to an embodiment of the present application according to a content recommendation command.

[0223] The controller 170 can execute the content recommendation menu 410 according to a content recommendation command to execute the content recommendation menu.

[0224] The content recommendation command can be an input in which the voice recognition button 236 of the remote control apparatus 200 is pressed for less than a preset time, but is not limited thereto.

[0225] The content recommendation menu 410 can include a menu for recommending other content related to specific content. The content recommendation menu 410 can include a music search item 411 for searching for music of specific content. According to an embodiment, the music search item 411 can be displayed as an item including text of "What is this music?"

[0226] A user input selecting the music search item 411 can be a music search command. That is, the controller 170 can execute the content recommendation menu to enter a state in which the music search command can be received.

[0227] The controller 170 can start audio recording using the microphone 151 at a point of time at which the content recommendation command is received. The point of time at which the controller 170 starts audio recording can be before the music search command is received.

[0228] Next, if a user input selecting the music search item 411 is received, the controller 170 can perform music search.

[0229] Accordingly, the controller 170 can display music information searched based on audio recorded from the point of time at which the content recommendation command is received.

[0230] Figure 11 FIG. 1 is a diagram illustrating a menu displayed by a mobile terminal connected with a display apparatus in order to control the display apparatus according to an embodiment of the present disclosure.

[0231] Figure 11 FIG. 2 can be a diagram illustrating a main menu 310 of an application program displayed on a display 551 of a mobile terminal 500.

[0232] The application program displayed on the display 551 of the mobile terminal 500 can be an application program for controlling various electronic apparatuses registered in the mobile terminal 500.

[0233] The main menu 310 of the application program can include a list of electronic apparatuses registered in the mobile terminal 500. The electronic apparatuses registered in the mobile terminal 500 can be electronic apparatuses that have been wirelessly connected with the mobile terminal 500 or are currently wirelessly connected with the mobile terminal 500 and can be controlled using the mobile terminal 500.

[0234] The electronic apparatuses registered in the mobile terminal 500 can include the display apparatus 100. Accordingly, the display apparatus 100 can be included in the list of electronic apparatuses registered in the mobile terminal 500.

[0235] On the other hand, the mobile terminal 500 can register not only the display apparatus 100 but also various electronic apparatuses such as a wireless vacuum cleaner, an optical wave oven, a dehumidifier, etc.

[0236] The list of electronic devices registered in the mobile terminal 500 can include an item 311 corresponding to the display device, an item 312 corresponding to the wireless vacuum cleaner, an item 313 corresponding to the light wave oven, and an item 314 corresponding to the dehumidifier, etc.

[0237] In addition, each of the items corresponding to the electronic devices can include action information of the electronic devices such as "on", "not connected", "on standby", etc.

[0238] If receiving an input of selecting the item 311 corresponding to the display device among the items included in the main menu 310, the mobile terminal 500 can display a menu for controlling the display device 100.

[0239] As for the menu for controlling the display device 100, refer to Figure 12 for a detailed description.

[0240] Figure 12 FIG. 4 is a diagram illustrating a form of a menu for controlling a display device of a mobile terminal connected with the display device according to an embodiment of the present application.

[0241] The menu 320 for controlling the display device 100 can include a remote control item 321, a sharing item 322, an application item 323, etc.

[0242] The remote control item 321 can be an item for remotely controlling the display device 100. The remote control item 321 can include an icon 3211 for controlling the microphone 522 of the mobile terminal to be turned on.

[0243] If receiving a user input of selecting the icon 3211 for controlling the microphone 522 of the mobile terminal to be turned on, the mobile terminal 500 can control the microphone 522 to be turned on.

[0244] The mobile terminal 500 can receive a voice command for remotely controlling the display device 100 through the microphone 522. The voice command can include a music search command.

[0245] That is, the mobile terminal 500 can receive a user input of selecting the icon 3211 for controlling the microphone 522 of the mobile terminal to be turned on, and enter a state capable of receiving a music search command.

[0246] On the other hand, if receiving a user input of selecting the icon 3211 for controlling the microphone 522 of the mobile terminal to be turned on, the mobile terminal 500 can transmit a signal notifying the same to the display device 100.

[0247] The display device 100 can start audio recording based on the signal received from the mobile terminal 500.

[0248] That is, if the mobile terminal 500 enters a state in which it can receive a music search command, the display device 100 can start audio recording. Thus, the point in time at which the display device 100 starts audio recording can be before the mobile terminal 500 receives a voice command.

[0249] If a voice command is received, the mobile terminal 500 can transmit information including the voice command to the display device 100.

[0250] The display device 100 can acquire the voice command as a music search command based on the information received from the mobile terminal 500. The display device 100 can perform music search based on the acquired music search command.

[0251] Thus, the display device 100 can display music information searched based on audio recorded from the point in time at which the user input selecting the icon 3211 for controlling the microphone 522 to be turned on is received.

[0252] On the other hand, if the remote control device 200 enters a state in which it can receive a music search command, the display device 100 can start audio recording.

[0253] For example, the remote control device 200 can receive a command for turning on the microphone 290. The command for turning on the microphone 290 can be input in which the voice recognition button 236 is pressed for more than a predetermined time and then released, but is not limited thereto.

[0254] The remote control device 200 can receive a voice command for the display device 100 through the microphone 290. The voice command can include a music search command.

[0255] That is, the remote control device 200 can enter a state in which it can receive a music search command for the display device 100.

[0256] On the other hand, if a command for turning on the microphone 290 is received, the remote control device 200 can transmit a signal notifying thereof to the display device 100.

[0257] The display device 100 can start audio recording based on the signal received from the remote control device 200.

[0258] That is, if the remote control device 200 enters a state in which it can receive a music search command, the display device 100 can record audio. Thus, the point in time at which the display device 100 starts audio recording can be before the remote control device 200 receives a voice command.

[0259] If a voice command is received, the remote control device 200 can transmit information including the voice command to the display device 100.

[0260] The display apparatus 100 can acquire a voice command as a music search command based on information received from the remote control apparatus 200. The display apparatus 100 can perform music search based on the acquired music search command.

[0261] Thereby, the display apparatus 100 can display music information searched based on the recorded audio from the mobile terminal 500 from a point in time at which the display apparatus 100 receives a user input selecting the icon 3211 for controlling the microphone 522 to be turned on.

[0262] Thereby, the display apparatus 100 can display music information searched based on the recorded audio from the remote control apparatus 200 from a point in time at which the display apparatus 100 receives a user input selecting the icon 3211 for controlling the microphone 522 to be turned on.

[0263] Figure 13 FIG. 1 is a diagram illustrating a screen displayed as the display apparatus receives a start-up utterance according to an embodiment of the present application.

[0264] The controller 170 can receive a start-up utterance for starting the display apparatus 100 through the microphone 151. The start-up utterance can be an instruction for starting the display apparatus 100 in order to acquire a voice command of a user. The start-up utterance can be a sentence or a word including a manufacturer name of the display apparatus 100, but the start-up utterance can be various.

[0265] In order to acquire a voice command of a user upon receiving the start-up utterance, the controller 170 can activate the microphone 151.

[0266] If the start-up utterance is received, the controller 170 can display at least one of a message 182 informing that a voice command of a user is being waited for or an icon 183 in standby for acquiring a voice command.

[0267] The message 182 informing of the standby for acquiring a voice command can be "Please speak now", but is not limited thereto.

[0268] The icon 183 for indicating the standby for acquiring a voice command can be a still icon, but can also be an icon moving such as rotating. In the case where the icon 183 for indicating the standby for acquiring a voice command is a moving icon, a user can know that the display apparatus 100 is in the standby for acquiring a voice command during the movement of the icon 183.

[0269] The controller 170 can act based on a voice command input after receiving the start-up utterance. Thereby, the controller 170 can act according to the voice command only in the case where a user explicitly plans to issue a voice command to the display apparatus 100.

[0270] The voice command can include a music search command. That is, the controller 170 can receive the start-up utterance and enter a state capable of receiving a music search command.

[0271] The controller 170 can start audio recording using the microphone 151 at the point of time when the start phrase is received. The point of time when the controller 170 starts audio recording can be before the music search command is received.

[0272] Next, if the music search command is received, the controller 170 can perform music search.

[0273] Thus, the controller 170 can display music information searched based on the audio recorded from the point of time when the start phrase is received.

[0274] Figure 14 FIG. 6 is a diagram illustrating a screen on which a menu including a music search item is displayed by a display apparatus according to an embodiment of the present application.

[0275] The menu including the music search item can include a quick setting menu 420. The quick setting menu 420 can be a menu for conveniently changing a setting state of the display apparatus 100 in content play.

[0276] The controller 170 can display the quick setting menu 420 based on a user input. The user can conveniently access items included in the quick setting menu 420 in content play. For example, the user input for displaying the quick setting menu 420 can be an input of pressing a setting button (not shown) of the remote control button 200, but is not limited thereto.

[0277] The quick setting menu 420 can include various items for facilitating convenience of the user. For example, the quick setting menu 420 can include an image mode item 421, a pixel brightness item 422, a sound mode item 423, a sound output item 424, a timer item 425, a vision protection mode item 426, a music search item 427, a screen-off item 428, etc.

[0278] The music search item 427 can be an item for searching for music of content displayed on the display 180.

[0279] The controller 170 can display a music search menu 430 based on a user input of selecting the music search item 427.

[0280] The music search menu 430 can include a background music search item 431 and a singing search item 432.

[0281] The background music search item 431 can be an item for searching for music included in content. The user input of selecting the background music search item 431 can be a music search command for searching for music included in content.

[0282] That is, the controller 170 can enter a state capable of receiving a music search command based on a user input selecting the music search item 427.

[0283] If a user input selecting the music search item 427 is received, the controller 170 can start audio recording. A time point at which the controller 170 starts audio recording can be before a music search command is received.

[0284] Next, if a user input selecting the background music search item 431 is received, the controller 170 can perform music search.

[0285] Thus, the controller 170 can display music information searched based on audio recorded from a time point at which a user input selecting the background music search item 431 is received.

[0286] A conventional display apparatus starts audio recording at a time point at which a music search command is received.

[0287] On the other hand, Figures 10-14 The display apparatus 100 of an embodiment can start audio recording before a time point at which a music search command is received.

[0288] Thus, Figures 10-14 The display apparatus 100 of an embodiment can start audio recording at an earlier time point than a time point at which a conventional display apparatus starts audio recording.

[0289] Thus, Figures 10-14 The display apparatus 100 of an embodiment can minimize a problem that music changes at a time point at which music search is planned before audio recording is started and thus music that a user wants to search for cannot be recorded.

[0290] In addition, Figures 10-14 The display apparatus 100 of an embodiment can take less time to display searched music information than a conventional display apparatus. Thus, the display apparatus 100 of an embodiment has an advantage of being capable of minimizing time taken to display searched music information.

[0291] On the other hand, a conventional display apparatus performs music search using one recorded audio. Thus, there is a problem that music information cannot be searched depending on quality of the recorded audio.

[0292] Thus, an object of a display apparatus 100 of an embodiment is to record audio a plurality of times and generate audio files, and search for music information based on at least one of the plurality of audio files.

[0293] That is, the display apparatus 100 of an embodiment can generate a plurality of audio files when performing Figure 7 step S13.

[0294] Referring to Figure 15 , a method of generating a plurality of audio files by the display apparatus 100 according to an embodiment of the present application will be described.

[0295] Figure 15 is a diagram illustrating a method of starting a plurality of recordings by the display apparatus according to an embodiment of the present application.

[0296] The controller 170 can record at a preset interval b from a time point a1 at which the user plans to record and generate a plurality of audio files. The preset interval b can be one second, but is not limited thereto.

[0297] The audio file generated by the controller 170 can be a PCM (Pulse Code Modulation) file. The PCM file can be a file generated by a method of converting an analog voice signal into a digital pulse signal. However, the audio file according to the present application is not limited to the PCM file.

[0298] Referring to Figure 15 , in order to generate a first audio file, the controller 170 can record audio during a preset time d from a time point a1 at which the user plans to record.

[0299] The time point a1 at which the user plans to record can be a time point a1 at which a state capable of receiving a music search command is entered.

[0300] The preset time d can be six seconds, but is not limited thereto.

[0301] In order to generate a second audio file, the controller 170 can record audio during a preset time d from a time point a2 one second after the time point at which the user plans to record.

[0302] In order to generate a third audio file, the controller 170 can record audio during a preset time d from a time point a3 two seconds after the time point at which the user plans to record.

[0303] In order to generate a fourth audio file, the controller 170 can record audio during a preset time d from a time point a2 three seconds after the time point at which the user plans to record.

[0304] On the other hand, the number of audio files generated by the controller 170 is not limited to four, and various numbers of audio files can be generated according to embodiments.

[0305] The first audio file and the second audio file can collectively include a waveform between a time point a2 at which audio recording for generating the second audio file is started and a time point a1' at which audio recording for generating the first audio file is ended.

[0306] The second audio file and the third audio file can collectively include a waveform between a time point a3 at which the audio recording for generating the third audio file is started and a time point a2' at which the audio recording for generating the second audio file is ended.

[0307] The third audio file and the fourth audio file can collectively include a waveform between a time point a4 at which the audio recording for generating the fourth audio file is started and a time point a3' at which the audio recording for generating the third audio file is ended.

[0308] As described above, the controller 170 can generate a plurality of audio files that are partially common.

[0309] That is, the display apparatus 100 according to an embodiment of the present application can generate a plurality of audio files including various sections of the same music.

[0310] Next, a method in which the controller 170 retrieves music when the display apparatus 100 generates a plurality of audio files will be described.

[0311] Figure 16 is a flowchart illustrating a method in which the display apparatus according to an embodiment of the present application retrieves music information using recorded files.

[0312] Figure 16 is Figure 7 is a flowchart in which step S19 is embodied when the audio recording is started at a plurality of time points and a plurality of audio files are generated.

[0313] The controller 170 can select an audio for music retrieval from among the plurality of audio files (S131).

[0314] The controller 170 can select an audio file for music retrieval based on at least one of a ratio of a silent section and an SNR (Signal to Noise Ratio) level of each of the plurality of audio files.

[0315] First, with reference to Figure 17 , a method in which the controller 170 selects an audio file for music retrieval based on a ratio of a silent section of each of the plurality of audio files will be described.

[0316] Figure 17 is a diagram for explaining a method in which the display apparatus according to an embodiment of the present application selects an audio file for music retrieval based on a ratio of a silent section of each of the plurality of audio files.

[0317] Figure 17 is a diagram illustrating audio waveforms of a first audio file 610, a second audio file 620, a third audio file 630, and a fourth audio file 640 from among the plurality of audio files 600 generated by the controller 170.

[0318] The controller 170 can screen the audio files in which the ratio of the silent section is less than a preset reference ratio.

[0319] The silent section can be a section in which no sound is recorded in the recorded audio. The silent section can be a section in which no waveform is output in the audio file.

[0320] The audio file in which the ratio of the silent section is the preset reference ratio or more can be an audio file in which it is difficult to perform music search.

[0321] Referring to Figure 17 , the ratio of the silent section of the first audio file 610 can be 10%. The ratio of the silent section of the second audio file 620 can be 30%. The ratio of the silent section of the third audio file 630 can be 55%. The ratio of the silent section of the fourth audio file 640 can be 60%.

[0322] The preset reference ratio can be 50%. Thus, the controller 170 can screen the first audio file 610 and the second audio file 620 in which the ratio of the silent section is 50% or less than the preset reference ratio as the audio files for music search.

[0323] The preset reference ratio is not limited to 50% and can be various according to embodiments.

[0324] Next, referring to Figure 18 , a method in which the controller 170 screens the audio files for music search based on the SNR (Signal to Noise Ratio) levels of the respective audio files will be described.

[0325] Figure 18 is a diagram for explaining a method in which a display device according to an embodiment of the present application screens the audio files for music search based on the SNR (Signal to Noise Ratio) levels of the respective audio files.

[0326] Figure 18 is a diagram showing the audio waveforms of the first audio file 610, the second audio file 620, the third audio file 630, and the fourth audio file 640 among the plurality of audio files 600 generated by the controller 170.

[0327] The controller 170 can display the music information based on the audio file in which the SNR level is the preset reference level or more among the plurality of audio files 600.

[0328] The SNR level can be a ratio of a signal of a recording content included in the audio file to a signal of a recording external noise. The smaller the ratio of the signal of the recording external noise, the higher the SNR level can be.

[0329] The controller 170 can screen the audio files 600 in which the SNR level is above a preset reference level.

[0330] The preset reference level can be set as a minimum level for obtaining a music search result, but is not limited thereto.

[0331] On the other hand, as the ratio of the signal of the recording external noise is higher, the waveform of the audio file can be more unstable.

[0332] Referring to Figure 18 The first audio file 610 has three intervals in which the amplitude is sharply increased and then decreased. The second audio file 620 has three intervals in which the amplitude is sharply increased and then decreased. In addition, the third audio file 630 has two intervals in which the amplitude is sharply increased and then decreased. On the other hand, the fourth audio file 640 has no interval in which the amplitude is sharply changed.

[0333] The fourth audio file 640 has a more stable waveform than the first audio file 610, the second audio file 620, and the third audio file 630.

[0334] Thus, it can be known that the ratio of the signal of the recording external noise of the fourth audio file 640 is smaller than those of the first audio file 610, the second audio file 620, and the third audio file 630.

[0335] The audio file screened by the controller 170 for music search can be the fourth audio file 640.

[0336] Again, it is explained Figure 16 .

[0337] The controller 170 can obtain an order of the audio files for music search based on at least one of the ratio of the silence interval and the SNR level of each of the screened audio files (S133).

[0338] For example, the controller 170 can obtain an audio file in which the ratio of the silence interval is the smallest among the screened audio files as an audio file used as a first priority in music search, and an audio file in which the ratio of the silence interval is next to the first audio file as an audio file used as a second priority in music search.

[0339] Alternatively, the controller 170 can acquire an audio file having the highest SNR level among the filtered audio files as an audio file used in music search as a first priority, and an audio file having an SNR level next to the first audio file as an audio file used in music search as a second priority.

[0340] In addition, the controller 170 can acquire the order of the audio files used in music search based on both the ratio of the silence section and the SNR level.

[0341] For example, the controller 170 can acquire an audio file having a higher SNR level among audio files having a ratio of the silence section less than 30% as an audio file used first in music search. Thus, even if an audio file having a ratio of the silence section of 30% or more is an audio file having the highest SNR level, it can not be used first in music search.

[0342] That is, the controller 170 sets a ratio of the silence section for acquiring the order used in search, and acquires an audio file having the highest SNR level among audio files having a ratio of the silence section lower than the set ratio as an audio file used first in music search.

[0343] In contrast, the controller 170 can acquire an audio file having a lower ratio of the silence section among audio files having an SNR level of a prescribed level or more as an audio file used first in music search.

[0344] That is, the controller 170 sets an SNR level for acquiring the order used in search, and acquires an audio file having the lowest ratio of the silence section among audio files having an SNR level of the set SNR level or more as an audio file used first in music search.

[0345] By considering both the ratio of the silence section and the SNR level to acquire the order of the audio files used in music search, it is possible to improve the likelihood of the controller 170 acquiring a music search result.

[0346] In addition, the method by which the controller 170 acquires the order of the audio files used in music search based on at least one of the ratio of the silence section and the SNR level of each of the filtered audio files can be various.

[0347] The controller 170 can acquire the type of content (S135).

[0348] The controller 170 can acquire the type of content from an external server (not shown) of the display apparatus 100.

[0349] The controller 170 can acquire the type of content from an external server (not shown) based on automatic content analysis, EPG mapping, etc.

[0350] The controller 170 can receive an automatic content analysis result from an external server (not illustrated). The automatic content analysis result can include a title, a type, a viewing information, a viewing time point of the content. For example, the controller 170 can transmit a video signal or an audio signal of the content to the external server (not illustrated). Thereby, the controller 170 can receive an automatic content analysis result based on the audio signal analysis from the external server (not illustrated).

[0351] The controller 170 can receive an EPG (Electronic Program Guide) mapping result from an external server (not illustrated). The EPG (Electronic Program Guide) mapping result can include a title, a type, a viewing time point, etc. of the content. For example, the controller 170 can transmit play information of the content to the external server (not illustrated). The play information of the content can include a channel of the content being played by the display device 100, a play time point. Thereby, the controller 170 can receive an EPG mapping result corresponding to the play information of the content from the external server (not illustrated).

[0352] In addition, the controller 170 can also transmit the recorded audio to the external server (not illustrated) to receive a type of the content.

[0353] In addition, the external server (not illustrated) can include a server (not illustrated) based on machine learning (Machine Learning).

[0354] The controller 170 can acquire whether a type of the content in the display device 100 is a music-related type (S137).

[0355] The controller 170 can acquire whether a type of the content acquired from the external server (not illustrated) is a music-related type.

[0356] The music-related type can include a concert, a classical, a jazz, a pop, etc.

[0357] The type not related to music can include a movie, a drama, a variety show, a documentary, etc.

[0358] In the case where the type of the content is not the music-related type, the controller 170 can remove a signal included in the recorded audio file except for the music signal (S138).

[0359] In the case where the type of the content is not the music-related type, the content can include a music signal and a signal other than music.

[0360] For example, the signal other than the music signal can be a voice signal of a dialogue included in the content. The controller 170 can improve the accuracy of the music search result by removing the signal other than the music signal included in the recorded audio file.

[0361] To this end, the display device 100 can store an algorithm for extracting a music signal.

[0362] The algorithm for extracting the background music can be an algorithm for extracting a music signal and a signal other than the music signal by analyzing a signal of an audio file.

[0363] In the case where the type of the content is not the music-related type, the controller 170 can search for the audio file from which the signal other than the music signal is removed in the original song search engine. Alternatively, in the case where the type of the content is the music-related type, the controller 170 can search for the audio file in the original song search engine (S139).

[0364] In the case where the type of the content is the music-related type, the audio file can not include the signal other than the music signal. Accordingly, a step of removing the signal other than the music signal included in the audio file can be omitted.

[0365] Thereby, the amount of calculation of the display device 100 can be reduced. In addition, the time required for the music search can be shortened.

[0366] The controller 170 can acquire whether the music search result is acquired from the original song search engine (S141).

[0367] The controller 170 can display the searched music information in the case where the music search result is acquired from the original song search engine (S147).

[0368] The music information searched from the original song search engine can include a singer, a title, an album, an album cover, an album release date, etc. of the original song. In addition, the controller 170 can display the accuracy of the searched music together in the case where the searched music information is displayed.

[0369] In the case where the search result can be acquired from the original song search engine, a step of performing a search using another music search engine such as the cover song search engine can be omitted. Thereby, the amount of calculation of the display device 100 can be reduced. In addition, the time required for displaying the music information can be shortened.

[0370] In the case where the music search result is not acquired from the original song search engine, the controller 170 can perform a search in the cover song search engine (S143).

[0371] In a case where the search result is not acquired from the original song search engine, the controller 170 can attempt music search in the cover song search engine.

[0372] In a case where the music search result is acquired from the cover song search engine, the controller 170 can display the searched music information (S147).

[0373] The music information searched in the cover song search engine can include a singer, a title, an album, an album cover, an album release date, etc. of the cover song. Also, in a case where the searched music information is displayed, the controller 170 can display the accuracy of the searched music together.

[0374] The controller 170 can provide information of the cover song by searching for music in the cover song search engine. Thereby, compared to a case where music search is performed only with the original song search engine, the controller 170 can provide diverse music information.

[0375] In a case where the search result is not acquired from the cover song search engine, the controller 170 can perform music search with the audio file for the next priority (S146).

[0376] In a case where the search result is not acquired from the cover song search engine, the controller 170 can repeat steps S135 to S147 with the audio file for the next priority. The controller 170 can repeat steps S139 to S148 until the music search result is acquired.

[0377] However, according to an embodiment, steps 135 to 137 with the audio file for the next priority can be omitted.

[0378] For example, in a case where the search result is not acquired from the cover song search engine, in a case where the content is a music-related type, the controller 170 can perform search in the original song search engine with the audio file for the next priority.

[0379] Or, in a case where the content is not a music-related type, the controller 170 can remove a signal other than a music signal included in the audio file for the next priority and perform search in the original song search engine.

[0380] All the audio files can be files generated from audio recorded at a time point adjacent to a time point at which the user plans to record.

[0381] Therefore, all the audio files can be files generated from audio recorded in a process in which the same content is played by the display device 100. Therefore, it can not be necessary to repeatedly acquire the type of steps 135 to 137.

[0382] SummaryFigure 16 The display apparatus 100 of an embodiment of the present application can display music information retrieved using a plurality of audio files and a search engine in sequence.

[0383] The display apparatus 100 of an embodiment of the present application can generate a plurality of audio files, and perform music search using at least one of the generated audio files. Thereby, compared to a case where one audio file is generated to perform music search, it is possible to minimize a problem of failure in obtaining a search result.

[0384] In addition, the display apparatus 100 of an embodiment of the present application can reduce a calculation amount for music search or shorten a time required to display a music search result, according to music included in content.

[0385] According to an embodiment of the present application, the aforementioned method can be implemented as a code readable by a processor in a medium in which a program is recorded. As an example of the medium readable by a processor, there are a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0386] In the display apparatus as described above, the configuration and method of the above-described embodiments are not limited to be used, but the embodiments can also be configured by selectively combining all or a part of each embodiment to be able to implement various modifications.

Claims

1. A display device, wherein, include: microphone; The controller receives a microphone-on command from a remote control device to control the microphone to turn on; as well as The display shows music information retrieved based on audio recorded using the microphone from the time the microphone is received.

2. The display device according to claim 1, wherein, The controller displays music information retrieved using audio recorded by the microphone, based on the time point from the moment the content recommendation command to the content recommendation menu is received.

3. The display device according to claim 1, wherein, If a startup message to activate the display device is received, the controller displays music information retrieved based on audio recorded using the microphone from the time the startup message was received.

4. The display device according to claim 1, wherein, The controller begins recording multiple times from the moment it receives the microphone turn-on command to generate multiple audio files; Displays music information retrieved based on at least one of the plurality of audio files.

5. The display device according to claim 1, wherein, The controller starts recording at preset intervals from the time it receives the microphone turn-on command to generate multiple audio files; Displays music information retrieved based on at least one of the plurality of audio files.

6. The display device according to claim 5, wherein, Display device. Displays music information based on audio files selected according to at least one of the ratio of the silence intervals of each of the plurality of said audio files and the signal-to-noise ratio level.

7. The display device according to claim 6, wherein, The controller displays the music information based on audio files in which the ratio of the silent intervals in a plurality of audio files is less than a preset baseline ratio.

8. The display device according to claim 6, wherein, The controller displays the music information based on a plurality of audio files whose signal-to-noise ratio level is above a preset benchmark level.

9. The display device according to claim 6, wherein, The controller obtains the order of audio files for music retrieval based on at least one of the ratio of the silence intervals of the respective filtered audio files and the signal-to-noise ratio level.

10. The display device according to claim 9, wherein, The controller selects the audio file with the smallest ratio of silence intervals from the filtered audio files as the first priority audio file in the music search, and selects the audio file with the second lowest ratio of silence intervals as the second priority audio file in the music search.

11. The display device according to claim 9, wherein, The controller selects the audio file with the highest signal-to-noise ratio (SNR) among the filtered audio files as the first priority audio file in the music search, and selects the audio file with the second highest SNR among the first audio files as the second priority audio file in the music search.

12. The display device according to claim 1, wherein, The controller obtains the type of content displayed on the monitor from an external server, and determines whether to remove signals other than music from the audio based on the type of content.

13. The display device according to claim 11, wherein, If the content is not of a music-related type, the controller displays music information retrieved from the audio after removing signals other than the music signal contained in the audio.

14. The display device according to claim 6, wherein, The controller performs a music search corresponding to the audio recorded using the microphone in at least one of a plurality of music search engines, including an original song search engine and a cover song search engine.

15. The display device according to claim 14, wherein, If the controller does not obtain music search results from the original song search engine, it performs a music search in the cover song search engine.

Images