Display device and display device control method

Abstract

A display device for providing a speech recognition function is presented. The display device comprises: a sensor for detecting the speech of a user; and a controller for verifying whether the speech of the user is an instruction for activating the speech recognition function, wherein the controller verifies the speech of the user by driving a plurality of verification engines in parallel, first outputs the result of the verification engine that has completed the verification of the speech of the user first, and can determine, on the basis of the results of the plurality of verification engines, whether the speech of the user is the instruction for activating the speech recognition function.

Description

Display device and method of controlling the display device

[0001] The present invention relates to a display device and a method for controlling the display device.

[0002] Recently, new form factors have been discussed in multimedia devices such as mobile phones and TVs. Form factors refer to the structured form of a product.

[0003] The growing importance of form factor innovation in the display industry stems from the increasing mobility of consumers, the rapid advancement of device convergence and smartness, and the growing need for form factors that can be used freely and conveniently regardless of the usage environment, moving away from typical form factors tailored to specific usage environments.

[0004] For example, vertical TVs are expanding, breaking the stereotype of viewing TV horizontally. These TVs, designed to accommodate the MZ generation's familiarity with mobile content, feature screen orientation adjustments. This allows users to conveniently view images from social media or shopping sites, and even read comments while watching videos. Vertical TVs are particularly advantageous when paired with smartphones via Near Field Communication (NFC)-based mirroring. When watching regular TV or movies, the TV can be rotated horizontally.

[0005] As another example, rollable TVs and foldable smartphones are similar in that they both use "flexible displays." A flexible display, as the name suggests, refers to electronic components that are flexible. For these displays to be flexible, they must first be thin. The substrate that receives information and converts it into light must be thin and flexible to ensure long-lasting performance without damage.

[0006] Flexibility means being able to withstand impact without significant impact. While a flexible display is being bent or folded, the joints are constantly under pressure. To prevent internal damage from this pressure, the display must be durable yet still be able to deform smoothly when subjected to pressure.

[0007] Flexible displays, for example, are implemented based on OLEDs. OLEDs are displays that utilize organic light-emitting materials. Organic materials are relatively more flexible than inorganic materials like metals. Furthermore, OLEDs have a thin substrate, giving them a competitive edge over other displays. Previously used LCD substrates required separate liquid crystals and glass, limiting their ability to be reduced in thickness.

[0008] As a new form factor for TVs, demand is growing for TVs that can be easily transported both indoors and outdoors. In particular, the recent coronavirus pandemic has led to increased time spent at home, driving demand for second TVs. Furthermore, the growing number of people going outdoors for activities like camping has driven demand for TVs in new form factors that are easy to carry and transport.

[0009] Unlike conventional TVs, these TVs are driven by software called an operating system, and provide a user interface through apps or programs.

[0010] Additionally, controlling the TV through voice commands or running TV apps and programs is becoming more common.

[0011] The operating system is determined by the model at the time of release, or the latest version available at the time of shipment is installed and delivered to the user. Due to the diverse form factors of TVs, various versions of the operating system are also available for TVs, and OS software updates are required over time after release.

[0012] As operating system software updates evolve, decisions and solutions regarding how to implement software (i.e., a verification engine) to validate user voice input for voice command functionality or voice recognition-based services are needed. This is because the results of user voice recognition depend on hardware conditions, such as the microphone and the distance between the microphone and speaker.

[0013] Meanwhile, the description below is not limited to TVs and applies to all devices that output images or sounds that support voice commands through voice recognition, and the term "display device" is used instead of "TV."

[0014] In order to solve the above-described problem, the present invention proposes a method for managing and using software (i.e., a verification engine) for verifying a user's voice for voice recognition.

[0015] More specifically, the present invention proposes a specific method for specifying settings for software (i.e., a verification engine) for verification of a start-up word to be used by a display device on a server and for driving or using the verification engine.

[0016] The problems to be solved by the present invention are not limited to the problems to be solved above, and other problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0017] A display device providing a voice recognition function is proposed, the display device including a sensor for detecting a user's voice; and a controller for verifying whether the user's voice is a command for activating a voice recognition function, wherein the controller verifies the user's voice by driving a plurality of verification engines in parallel, and outputs first the results of a verification engine that has first completed verification of the user's voice, and can determine whether the user's voice is a command for activating the voice recognition function or not based on the results of the plurality of verification engines.

[0018] A system for voice recognition, comprising a display device; and an activation word verification server, wherein the display device comprises a sensor for detecting a user's voice; and a controller for verifying whether the user's voice is a command for activating a voice recognition function, wherein the activation word verification server receives the user's voice from the display device and includes a controller for verifying whether the user's voice is a command for activating a voice recognition function, wherein the controller of the display device controls a plurality of verification engines of the display device or the activation word verification server to run in parallel to verify the user's voice, and determines whether the user's voice is a command for activating the voice recognition function or not based on a result of a verification engine embedded in the display device and a result of a verification engine embedded in the activation word verification server.

[0019] A method for a voice recognition function is proposed, the method being performed by a display device including a sensor for detecting a user's voice; and a controller for verifying whether the user's voice is a command for activating a voice recognition function, the method including: a step of recognizing the user's voice; a step of initiating a verification procedure for the user's voice by driving a plurality of verification engines in parallel; and a step of first outputting a result of a verification engine that has first completed verification of the user's voice, and determining whether the user's voice is a command for activating the voice recognition function or not based on the results of the plurality of verification engines.

[0020] The above problem solving methods are only some of the embodiments of the present invention, and various embodiments reflecting the technical features of the present invention can be derived and understood by a person having ordinary knowledge in the relevant technical field based on the detailed description of the present invention described below.

[0021] The present invention has the following effects.

[0022] The present invention can efficiently perform verification of a start-up word that initiates a voice recognition service, since it can separately manage a voice recognition verification engine even when a software update of a display device is performed.

[0023] In addition, the present invention can reduce the time required for verification of a trigger word by operating multiple verification engines in parallel for verification of a trigger word that initiates a voice recognition service.

[0024] In addition, the present invention can reduce the time required for verification of a trigger word by operating multiple verification engines belonging to different entities in parallel for verification of a trigger word that initiates a voice recognition service, and can provide scalability for an algorithm for a verification engine.

[0025] The accompanying drawings, which are included as part of the detailed description to aid in understanding the present invention, provide embodiments of the present invention and, together with the detailed description, explain the technical idea of ​​the present invention.

[0026] Figure 1 is a block diagram for explaining each component of the display device.

[0027] FIG. 2 is a drawing showing a display device according to one embodiment of the present invention.

[0028] Figure 3 illustrates a procedure for recognizing a starting word or starting a voice recognition service according to the present invention.

[0029] Figure 4 illustrates a procedure for recognition and verification of a starting word for a voice recognition service according to the present invention.

[0030] Figure 5 illustrates a flowchart of a procedure for recognition and verification of a starting word for a voice recognition service according to the present invention.

[0031] Figure 6 illustrates a flowchart of a procedure for recognition and verification of a starting word for a voice recognition service according to the present invention.

[0032] Figure 7 illustrates a flowchart of a procedure for recognition and verification of a starting word for a voice recognition service according to the present invention.

[0033] FIG. 8 illustrates a block diagram of a controller for recognition and verification of a starting word for a voice recognition service according to the present invention.

[0034] FIG. 9 illustrates a block diagram of a system for recognition and verification of a starting word for a voice recognition service according to the present invention.

[0035] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0036] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0037] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0038] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0039] In this application, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0040]

[0041] In the following description, the display device (100) is referred to as such, but the display device may be referred to by various names, such as a TV or a multimedia device, and the scope of the invention is not limited to the name.

[0042]

[0043] FIG. 1 is a block diagram for explaining each component of a display device (100) according to one embodiment of the present invention.

[0044] The display device (100) may include a broadcast receiving unit (110), an external device interface unit (171), a network interface unit (172), a storage unit (140), a user input interface unit (173), an input unit (130), a control unit (180), a display module (150), an audio output unit (160), and / or a power supply unit (190).

[0045] The broadcast receiving unit (110) may include a tuner unit (111) and a demodulator unit (112).

[0046] Meanwhile, unlike the drawing, the display device (100) may include only the external device interface unit (171) and the network interface unit (172) among the broadcast receiving unit (110), the external device interface unit (171), and the network interface unit (172). That is, the display device (100) may not include the broadcast receiving unit (110).

[0047] The tuner unit (111) can select a broadcast signal corresponding to a channel selected by the user or all previously stored channels among broadcast signals received through an antenna (not shown) or a cable (not shown). The tuner unit (111) can convert the selected broadcast signal into an intermediate frequency signal or a baseband video or audio signal.

[0048] For example, the tuner unit (111) can convert the selected broadcast signal into a digital IF signal (DIF) if it is a digital broadcast signal, and can convert it into an analog baseband video or audio signal (CVBS / SIF) if it is an analog broadcast signal. That is, the tuner unit (111) can process a digital broadcast signal or an analog broadcast signal. The analog baseband video or audio signal (CVBS / SIF) output from the tuner unit (111) can be directly input to the control unit (180).

[0049] Meanwhile, the tuner unit (111) can sequentially select broadcast signals of all stored broadcast channels through the channel memory function among the received broadcast signals and convert them into intermediate frequency signals or baseband video or audio signals.

[0050] Meanwhile, the tuner unit (111) may be equipped with multiple tuners to receive broadcast signals of multiple channels. Alternatively, a single tuner that simultaneously receives broadcast signals of multiple channels is also possible.

[0051] The demodulation unit (112) can perform a demodulation operation by receiving a digital IF signal (DIF) converted by the tuner unit (111). The demodulation unit (112) can output a stream signal (TS) after performing demodulation and channel decoding. At this time, the stream signal can be a signal in which a video signal, an audio signal, or a data signal is multiplexed.

[0052] The stream signal output from the demodulation unit (112) can be input to the control unit (180). The control unit (180) can perform demultiplexing, image / audio signal processing, etc., and then output an image through the display module (150) and output an audio through the audio output unit (160).

[0053] The sensing unit (120) refers to a device that detects changes within the display device (100) or detects external changes. For example, it may include at least one of a proximity sensor, an illumination sensor, a touch sensor, an infrared sensor (IR sensor), an ultrasonic sensor, an optical sensor (e.g., a camera), a voice sensor (e.g., a microphone), a battery gauge, and an environmental sensor (e.g., a hygrometer, a thermometer, etc.).

[0054] The control unit (180) can check the status of the display device (100) based on the information collected from the sensing unit (120), and if a problem occurs, can notify the user of it or control it to maintain the best condition by adjusting it on its own.

[0055] In addition, the content, picture quality, size, etc. of the image provided to the display module (150) can be controlled differently depending on the viewer detected by the sensing unit or the surrounding lighting, etc., to provide an optimal viewing environment. As smart display devices advance, the functions installed in the display devices increase, and the sensing unit (20) also increases along with them.

[0056] The input unit (130) may be provided on one side of the main body of the display device (100). For example, the input unit (130) may include a touch pad, a physical button, etc. The input unit (130) may receive various user commands related to the operation of the display device (100) and transmit a control signal corresponding to the input command to the control unit (180).

[0057] Recently, as the size of the bezel of the display device (100) has become smaller, the number of display devices (100) with a minimal input unit (130) in the form of a physical button exposed to the outside has increased. Instead, a minimum number of physical buttons are positioned on the back or side, and user input can be received through a remote control device (200) via a touchpad or a user input interface unit (173) described later.

[0058] The storage unit (140) may store programs for each signal processing and control within the control unit (180), or may store signal-processed video, audio, or data signals. For example, the storage unit (140) may store application programs designed for the purpose of performing various tasks that can be processed by the control unit (180), and may selectively provide some of the stored application programs upon request from the control unit (180).

[0059] Programs stored in the storage unit (140) are not particularly limited as long as they can be executed by the control unit (180). The storage unit (140) may also perform a function for temporarily storing video, audio, or data signals received from an external device through the external device interface unit (171). The storage unit (140) may store information regarding a specific broadcast channel through a channel memory function such as a channel map.

[0060] Although the storage unit (140) of FIG. 1 is provided separately from the control unit (180), the scope of the present invention is not limited thereto, and the storage unit (140) may be included within the control unit (180).

[0061] The storage unit (140) may include at least one of volatile memory (e.g., DRAM, SRAM, SDRAM, etc.) or non-volatile memory (e.g., flash memory, hard disk drive (HDD), solid-state drive (SSD), etc.).

[0062] The display module (150) can generate a driving signal by converting a video signal, data signal, OSD signal, control signal, etc. processed by the control unit (180) or a video signal, data signal, control signal, etc. received from the interface unit (171). The display module (150) can include a display panel having a plurality of pixels.

[0063] The plurality of pixels provided on the display panel may have RGB sub-pixels. Alternatively, the plurality of pixels provided on the display panel may have RGBW sub-pixels. The display module (150) can convert image signals, data signals, OSD signals, control signals, etc. processed by the control unit (180) to generate driving signals for the plurality of pixels.

[0064] The display module (150) can be a PDP (Plasma Display Panel), an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diode), a flexible display module, etc., and may also be a 3D display module. The 3D display module (150) can be divided into a glasses-free type and a glasses type.

[0065] The display device (100) includes a display module that occupies most of the front surface area and a case that covers the rear side of the display module and packages the display module.

[0066] Recently, display devices (100) can utilize display modules (150) that can be bent, such as LEDs (Light Emitting Diodes) or OLEDs (Organic Light Emitting Diodes), to implement a curved screen rather than a flat screen.

[0067] Previously, LCDs were primarily used because they were unable to emit light on their own, so they relied on backlight units for light. Backlight units are devices that evenly distribute light from a light source to the liquid crystal display (LCD) located at the front. As backlight units have become thinner, thinner LCDs have become possible. However, it has been difficult to implement backlight units using flexible materials, and when the backlight unit is bent, it becomes difficult to supply light evenly to the LCD, causing screen brightness to fluctuate.

[0068] On the other hand, in the case of LEDs or OLEDs, since each element forming a pixel emits light on its own, it is possible to implement a curved display without using a backlight unit. In addition, since each element emits light on its own, even if the positional relationship with neighboring elements changes, its own brightness is not affected, so a curved display module (150) can be implemented using LEDs or OLEDs.

[0069] OLED (Organic Light-Emitting Diodes) panels made their debut in the mid-2010s and are rapidly replacing LCDs in the small- and medium-sized display market. OLEDs utilize the self-luminous phenomenon, where fluorescent organic compounds emit light when current flows through them. Their image quality response speed is faster than that of LCDs, resulting in virtually no afterimages when displaying video.

[0070] OLED uses three types of fluorescent organic compounds with self-luminous functions: red, green, and blue. It is a luminescent display product that utilizes the phenomenon in which electrons injected from the cathode and anode and positively charged particles combine within the organic material to emit light on their own, so there is no need for a backlight (backlight device) that reduces color.

[0071] LED (Light Emitting Diode) panels are a technology that uses one LED element as one pixel, and can reduce the size of the LED element compared to the past, enabling the implementation of a flexible display module (150). Devices previously referred to as LED TVs used LEDs as a light source for a backlight unit that supplied light to an LCD, and the LEDs themselves did not form a screen.

[0072] The display module includes a display panel, a coupling magnet positioned on a rear surface of the display panel, a first power supply unit, and a first signal module. The display panel may include a plurality of pixels (R, G, B). The plurality of pixels (R, G, B) may be formed in each area where a plurality of data lines and a plurality of gate lines intersect. The plurality of pixels (R, G, B) may be arranged or arranged in a matrix form.

[0073] For example, the plurality of pixels (R, G, B) may include a red (Red, hereinafter referred to as 'R') sub-pixel, a green (Green, 'G') sub-pixel, and a blue (Blue, 'B') sub-pixel. The plurality of pixels (R, G, B) may further include a white (White, hereinafter referred to as 'W') sub-pixel.

[0074] The side of the display module (150) that displays an image can be referred to as the front or front. When the display module (150) displays an image, the side from which the image cannot be observed can be referred to as the rear or back.

[0075] Meanwhile, the display module (150) is configured as a touch screen and can be used as an input device in addition to an output device.

[0076] The audio output unit (160) receives a signal processed by the control unit (180) and outputs it as voice.

[0077] The interface unit (170) serves as a passageway for various types of external devices connected to the display device (100). The interface unit may include not only a wired method for transmitting and receiving data through a cable, but also a wireless method using an antenna.

[0078] The interface unit (170) may include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module, an audio I / O (Input / Output) port, a video I / O (Input / Output) port, and an earphone port.

[0079] As an example of a wireless method, the broadcast receiving unit (110) described above may be included, and may include not only broadcast signals but also mobile communication signals, short-range communication signals, wireless Internet signals, etc.

[0080] The external device interface unit (171) can transmit or receive data with a connected external device. To this end, the external device interface unit (171) may include an A / V input / output unit (not shown).

[0081] The external device interface unit (171) can be connected to external devices such as a DVD (Digital Versatile Disk), Blu-ray, game device, camera, camcorder, computer (laptop), set-top box, etc., via wired / wireless connection, and can also perform input / output operations with the external devices.

[0082] In addition, the external device interface unit (171) can establish a communication network with various remote control devices (200), and receive a control signal related to the operation of the display device (100) from the remote control device (200), or transmit data related to the operation of the display device (100) to the remote control device (200).

[0083] The external device interface unit (171) may include a wireless communication unit (not shown) for short-range wireless communication with other electronic devices. Through this wireless communication unit (not shown), the external device interface unit (171) can exchange data with an adjacent mobile terminal. In particular, in mirroring mode, the external device interface unit (171) can receive device information, running application information, application images, etc. from the mobile terminal.

[0084] The network interface unit (172) may provide an interface for connecting the display device (100) to a wired / wireless network, including the Internet. For example, the network interface unit (172) may receive content or data provided by the Internet, a content provider, or a network operator via a network. Meanwhile, the network interface unit (172) may include a communication module (not shown) for connection to a wired / wireless network.

[0085] The external device interface unit (171) and / or the network interface unit (172) may include a communication module for short-range communication such as Wi-Fi (Wireless Fidelity), Bluetooth, Bluetooth Low Energy (BLE), Zigbee, and NFC (Near Field Communication), a communication module for cellular communication such as LTE (long-term evolution), LTE-A (LTE Advance), CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), and WiBro (Wireless Broadband).

[0086] The user input interface unit (173) can transmit a signal input by the user to the control unit (180), or transmit a signal from the control unit (180) to the user. For example, it can transmit / receive a user input signal such as power on / off, channel selection, screen setting, etc. from a remote control device (200), or transmit a user input signal input from a local key (not shown) such as a power key, a channel key, a volume key, a setting value, etc. to the control unit (180), or transmit a user input signal input from a sensor unit (not shown) that senses a user's gesture to the control unit (180), or transmit a signal from the control unit (180) to the sensor unit.

[0087] The control unit (180) may include at least one processor, and may control the overall operation of the display device (100) using the processor included therein. Here, the processor may be a general processor such as a central processing unit (CPU). Of course, the processor may be a dedicated device such as an ASIC or another hardware-based processor.

[0088] The control unit (180) can demultiplex a stream input through a tuner unit (111), a demodulator unit (112), an external device interface unit (171), or a network interface unit (172), or process demultiplexed signals to generate and output a signal for video or audio output.

[0089] The image signal processed by the control unit (180) can be input to the display module (150) and displayed as an image corresponding to the image signal. In addition, the image signal processed by the control unit (180) can also be input to an external output device through the external device interface unit (171).

[0090] The voice signal processed in the control unit (180) can be output as sound to the audio output unit (160). In addition, the voice signal processed in the control unit (180) can be input to an external output device through the external device interface unit (171). In addition, the control unit (180) can include a demultiplexing unit, an image processing unit, etc.

[0091] In addition, the control unit (180) can control the overall operation within the display device (100). For example, the control unit (180) can control the tuner unit (111) to select (tuning) a broadcast corresponding to a channel selected by the user or a previously stored channel.

[0092] In addition, the control unit (180) can control the display device (100) by a user command or an internal program input through the user input interface unit (173). Meanwhile, the control unit (180) can control the display module (150) to display an image. At this time, the image displayed on the display module (150) may be a still image or a moving image, and may be a 2D image or a 3D image.

[0093] Meanwhile, the control unit (180) can cause a predetermined 2D object to be displayed within an image displayed on the display module (150). For example, the object can be at least one of a connected web screen (newspaper, magazine, etc.), an EPG (Electronic Program Guide), various menus, widgets, icons, still images, videos, and text.

[0094] Meanwhile, the control unit (180) may modulate and / or demodulate a signal using the Amplitude Shift Keying (ASK) method. Here, the Amplitude Shift Keying (ASK) method may mean a method of modulating a signal by varying the amplitude of a carrier wave according to a data value, or of restoring an analog signal to a digital data value according to the amplitude of the carrier wave.

[0095] For example, the control unit (180) can modulate a video signal using the amplitude shift keying (ASK) method and transmit it through a wireless communication module.

[0096] For example, the control unit (180) can demodulate and process an image signal received through a wireless communication module using the amplitude shift keying (ASK) method.

[0097] Through this, the display device (100) can easily transmit and receive signals with other adjacently placed video display devices without using a unique identifier such as a MAC address (Media Access Control Address) or a complex communication protocol such as TCP / IP.

[0098] Meanwhile, the display device (100) may further include a camera (not shown). The camera can capture images of the user. The camera can be implemented with a single camera, but is not limited thereto, and may also be implemented with multiple cameras. Meanwhile, the camera can be embedded in the display device (100) above the display module (150) or can be separately positioned. Image information captured by the camera can be input to the control unit (180).

[0099] The control unit (180) can recognize the user's location based on the image captured by the camera. For example, the control unit (180) can determine the distance (z-axis coordinate) between the user and the display device (100). In addition, the control unit (180) can determine the x-axis coordinate and y-axis coordinate within the display module (150) corresponding to the user's location.

[0100] The control unit (180) can detect the user's gesture based on the image captured from the camera unit, or the signal detected from the sensor unit, or a combination thereof.

[0101] The power supply unit (190) can supply power to the entire display device (100). In particular, it can supply power to a control unit (180) that can be implemented in the form of a system on chip (SOC), a display module (150) for displaying images, and an audio output unit (160) for audio output.

[0102] Specifically, the power supply unit (190) may be equipped with a converter (not shown) that converts AC power into DC power and a Dc / Dc converter (not shown) that converts the level of DC power.

[0103] Meanwhile, the power supply unit (190) receives power from an external source and distributes power to each component. The power supply unit (190) may utilize a method of supplying AC power by directly connecting to an external power source, or may include a power supply unit (190) that can be recharged and used, including a battery.

[0104] The former requires a wired cable connection, making movement difficult or limited. The latter offers freedom of movement, but increases weight and volume by the battery. Charging requires a direct connection to a power cable or a charging cradle (not shown) for a set period of time.

[0105] The charging cradle can be connected to the display device through an externally exposed terminal, or the built-in battery can be charged when brought into proximity using a wireless method.

[0106] The remote control device (200) can transmit user input to the user input interface unit (173). To this end, the remote control device (200) can use Bluetooth, RF (Radio Frequency) communication, infrared (Infrared Radiation) communication, UWB (Ultra-wideband), ZigBee, etc. In addition, the remote control device (200) can receive images, voices, or data signals output from the user input interface unit (173) and display or output the same as voice on the remote control device (200).

[0107] Meanwhile, the above-described display device (100) may be a digital broadcast receiver capable of receiving fixed or mobile digital broadcasts.

[0108] Meanwhile, the block diagram of the display device (100) illustrated in FIG. 1 is only a block diagram for one embodiment of the present invention, and each component of the block diagram may be integrated, added, or omitted depending on the specifications of the display device (100) actually implemented.

[0109] That is, two or more components may be combined into a single component, or a single component may be subdivided into two or more components, as needed. Furthermore, the functions performed by each block are intended to illustrate embodiments of the present invention, and their specific operations or devices do not limit the scope of the present invention.

[0110]

[0111] Figure 2 is a drawing illustrating a display device according to one embodiment of the present invention. Any description that overlaps with the above description will be omitted.

[0112] Referring to FIG. 2, the display device (100) is configured such that a display module (150) is housed inside a housing (210). At this time, the housing (210) includes an upper case (210a) and a lower case (210b), and the upper case (210a) and the lower case (210b) can be made in a structure that opens and closes.

[0113] In one embodiment, an audio output unit (160) may be included in the upper case (210a) of the display device (100), and a control unit (180), such as a main board, a power board, a power supply unit (190), a battery, an interface unit (170), a sensing unit (120), and an input unit (130, including a local key), may be housed in the lower case (210b). Here, the interface unit (170) may include a Wi-Fi module, a Bluetooth module, an NFC module, and the like for communication with an external device, and the sensing unit (120) may include a light sensor and an IR sensor.

[0114] In one embodiment, the display module (150) may include a DC-DC board, a sensor, and an LVDS (low voltage differential signaling) conversion board.

[0115] Additionally, in one embodiment, the display device (100) may further include four detachable legs (220a, 220b, 220c, 220d). Here, the four legs (220a, 220b, 220c, 220d) may be attached to the lower case (210b) to allow the display device (100) to be spaced from the floor.

[0116] The display device according to Fig. 2 has a movable feature.

[0117]

[0118] Figure 3 illustrates a procedure for recognizing a starting word or starting a voice recognition service according to the present invention.

[0119] (a) to (d) represent sequential procedures.

[0120] To initiate a voice recognition service for the display device (100), the user says “Hi, LG” (Fig. 3 (a)). This is a pre-arranged activation word for the voice recognition service, but is merely an example and may be set to a different activation word.

[0121] The display device (100) outputs on the display that voice command-based control (or voice recognition service) is initiated ((b) of FIG. 3), thereby informing the user that the user has spoken or must speak a voice command or that the display device (100) is ready to recognize the user's voice command.

[0122] Accordingly, the user inputs a voice command by saying “search big bang theory” (Fig. 3 (c)).

[0123] The display device (100) recognizes the user's voice command and performs a search based on it ((d) of FIG. 3).

[0124] A voice recognition service of the display device (100) is provided in the manner described with reference to FIG. 3. However, in order to enter the state of (b) of FIG. 3, it is necessary to verify whether the user's spoken activation word is a preset activation word (“Hi, LG”), and the user's utterance must be verified as being a preset activation word.

[0125] This is because, since the display device (100) is for viewing images, if a voice recognition service is initiated even though the user did not actually initiate the voice recognition service due to a misrecognition result, it may act as a factor that interferes with the user's viewing environment.

[0126]

[0127] Figure 4 illustrates a procedure for recognition and verification of a starting word for a voice recognition service according to the present invention.

[0128] Since (a) and (b) of Fig. 4 are identical to (a) and (b) of Fig. 3, their descriptions are omitted.

[0129] When a user's speech or voice is input, the display device (100) performs a trigger word recognition (S101) and a trigger word verification (S102) procedure. After the user's speech or voice is verified as a trigger word according to the trigger word verification (S102) procedure, a voice recognition service such as that shown in (b) of Fig. 4 is initiated.

[0130] Below, the startup verification procedure is described in more detail in relation to updating the operating system software (hereinafter referred to as “software”) of the display device.

[0131]

[0132] Figure 5 illustrates a flowchart of a procedure for recognition and verification of a starting word for a voice recognition service according to the present invention.

[0133] Referring to FIG. 5, two display devices (1001, 1002) are illustrated, but this is only an example, and the procedure described below may be applied through one or more display devices and a server (300).

[0134] The display device (1001) and the display device (1002) may be general household TVs, and as can be seen from the software version, the display device (1002) is a newer model that was released slightly later than the display device (1003).

[0135] Each display device is configured to use a pre-configured startup verification engine based on hardware and software version information. However, the issue of how the startup verification engine will be updated to the latest software version must be addressed.

[0136] As shown in Fig. 5, even after a software version update for each display device, the startup word verification engine used prior to the software update is set to be used as is. However, this is only an example, and the startup word verification engine to be used may be determined based on the startup word verification engine information (S551, S552) provided by the server, such as using an improved version of the startup word verification engine used prior to the software update.

[0137] Returning to Figure 5, the display devices (1001, 1002) each receive a voice input from the user to initiate a voice recognition service and perform activation word verification accordingly (S501, S502). The display devices (1001, 1002) each use activation word verification engines (hereinafter referred to as “verification engines”) A and B+C for activation word verification. “B+C” means that two verification engines, B and C, are used.

[0138] Software updates for each display device (1001, 1002) are completed by the server (300) (software versions are changed from 2.00.13, 3.04.31 to 13.00.01) (S511 and S521, S512 and S522). Software version 13.00.01 is the latest software version, and it is assumed that display devices designed or set up based on the software version use the startup word verification engine D.

[0139] Then, the remote voice recognition function is re-executed for each display device (1001, 1002) (S531, S532).

[0140] The display device (1001, 1002) may transmit a request for startup verification engine information to the server (300) (S541, S542). The request for startup verification engine information is intended to request the server (300) about the startup verification engine to be used by the display device (1001, 1002). The request for startup verification engine information may include software version information of the display device (1001, 1002), hardware information, and server version information.

[0141] Accordingly, the server (300) can transmit the activation word verification engine information to be used by each display device (1001, 1002) to the display devices (1001, 1002) (S551, S552).

[0142] The information about the activation word verification engine to be used by each display device (1001, 1002) depends on the hardware information of the display device. That is, the information about the activation word verification engine to be used by the display device can be determined based on the hardware information of the display device. Speech recognition inevitably depends on the performance of the voice detector (i.e., a sensor such as a microphone), and the position of the microphone, the arrangement structure of the microphone and speaker, the distance between the microphone and speaker, etc., can affect the speech recognition results depending on each display device (i.e., hardware information). Therefore, even if the software of the display device is updated to the latest version, the verification engine supported by the latest software is not used, but rather, the verification engine based on at least the hardware information of the display device is set to be used. In addition, the software version of each display device (1001, 1002) can also be used to determine the information about the activation word verification engine to be used by the display device (1001, 1002).

[0143]

[0144] The information about the startup verification engine to be used by the display device may be preset. Alternatively, the information about the startup verification engine to be used by the display device may be dynamically determined by the server based on at least the information included in the request for startup verification engine information.

[0145] The display device (1001, 1002) determines the verification engine based on the startup word verification engine information received from the server (300) (S561, S562). For example, if the received startup word verification engine information indicates verification engine A, the display device determines verification engine A as the verification engine to use. That is, the latest software version (13.00.01) of the display device sets the display device to use verification engine D, but based on the request for startup word verification engine information and the exchange or transmission / reception of startup word verification engine information with the server (300), the display device is set to use verification engines A and B+C that are most suitable for the display device (1001, 1002). In conclusion, the verification engine supported by the latest version of the software of the display device (1001, 1002) and the verification engine to be used by the display device (1001, 1002) may be different from each other.

[0146] The display device (1001, 1002) uses the determined verification engine for activation word verification (S571, S572).

[0147]

[0148] Figure 6 illustrates a flowchart of a procedure for recognition and verification of a starting word for a voice recognition service according to the present invention.

[0149] Referring to FIG. 6, two display devices (1003, 1004) are illustrated, but this is only an example, and the procedure described below may be applied through one or more display devices and a server (300).

[0150] The display device (1003) may be a general indoor TV, and the display device (1004) may be an outdoor TV.

[0151] As can be seen from the software version, the display device (1003) is a newer model that was released slightly later than the display device (1004).

[0152] Fig. 6 is different from Fig. 5 in hardware information and software version information of each display device, and the rest is the same as Fig. 5.

[0153] Meanwhile, as described with respect to FIG. 5, the verification engine used by the display device (1003, 1004) after the software update is the same as the verification engine used before the software update. This is only an example, and the verification engine given in the startup verification engine information (S651, S652) is used.

[0154]

[0155] Figure 7 illustrates a flowchart of a procedure for recognizing and verifying a trigger word for a voice recognition service according to the present invention. The procedure of Figure 7 is performed by a display device (100). The display device (100) includes a sensor for detecting a user's voice; and a controller for verifying whether the user's voice is a command for activating a voice recognition function. The display device (100) may also include a display (150).

[0156] The display device (100) can recognize a trigger word for a voice recognition service by recognizing the user's voice (S710).

[0157] Then, the display device (100) executes the activation word verification engine and issues a ticket (S720).

[0158] Tickets are generated for each recognized trigger word. That is, the same ticket is used for each trigger word. Simply put, tickets can be set to have a positive integer value. Furthermore, tickets are generated for the number of validation engines to be used.

[0159] Therefore, assuming that one activation word is recognized and there are two (2) verification engines received from the server (300) as described in FIGS. 5 and 6, two tickets having a value of “1” are issued. In addition, in order to manage the number of tickets having a value of “1” issued, a verification counter value is defined, and the verification counter value can be set to a value equal to the “number of tickets” having a value of one issued (i.e., in the above example, the verification counter value is 2).

[0160] If the activation words are entered and recognized consecutively, tickets with different ticket values ​​will be issued for each activation word. A ticket with a value of "1" will be issued for the first activation word, a ticket with a value of "2" will be issued for the second activation word, and so on.

[0161] The display device (100) can perform startup word verification by executing startup word verification engines in parallel (S730-1, S730-2, …, S730-N). In Fig. 7, N verification engines are illustrated, where N is an integer greater than or equal to 1. For reference, if there is only one verification engine, “parallel execution” will not occur.

[0162] Each verification engine can output verification results using a verification algorithm for the recognized trigger word. The verification results obtained from the verification engine can be stored and managed, as shown in the example in Table 1 below.

[0163] Verification results are determined by collecting the results of one or more verification engines for tickets with the same value, i.e., for a single trigger word. Accordingly, the display device (100) can verify whether the judgment results for all issued tickets have been generated or collected (S740). The aforementioned verification counter value can be utilized for this purpose. The verification counter value set when issuing a ticket is decremented by 1 each time a verification result from each verification engine is generated or collected, and the result of S740 can be determined by whether the verification counter value reaches 0.

[0164] The display device (100) can collect the verification results determined by each verification engine and determine the verification results (S750). At this time, if multiple verification engines are used, the display device (100) can derive the verification results by applying individual weights to the results of each verification engine. The individual weights applied to the results of each verification engine may be the same, different, or at least some may be the same and the rest may be different.

[0165] The verification results are divided into Pass and No pass or fail, and the thresholds or reference values ​​for determining this can be set and implemented in various ways.

[0166] The display device (100) can transmit the verification result to a storage such as a memory for managing the verification result (S760).

[0167] Meanwhile, if the verification result is 'pass', the verification result can be transmitted to another service, app, program within the display device (100), another service supported by the display device (100), or a device related thereto.

[0168] The following table shows that the display device (100) stores and manages verification results related to the startup word verification engine in a queue data structure.

[0169] Ticket: “1”VeriCnt: 1VeriRst: K1Ticket: “2”VeriCnt: 2VeriRst: K2Ticket: “1”VeriCnt: 0VeriRst: K3…

[0170] Ticket represents the ticket value, VeriCnt represents the verification counter value, and VeriRst represents the verification result derived from the verification engine.

[0171]

[0172] Fig. 8 illustrates a block diagram of a control unit (180) of a display device (100) for recognition and verification of a spurious word for a voice recognition service according to the present invention. The "control unit" may be referred to as a "controller." Furthermore, a brief description will be given below with reference to the block diagram of Fig. 8, and any overlapping portions with the contents of the present invention previously described with reference to Figs. 3 to 7 will be referred to the corresponding descriptions and omitted below.

[0173] The controller (180) may include a voice recognizer (181), a verification executor (182), a verification result judger (183), and a verification result transmitter (184).

[0174] A voice recognizer (181) can recognize a user's voice. The voice recognizer (181) may be composed of a sensor such as a microphone.

[0175] The verification executor (182) can verify whether the recognized user's voice corresponds to a trigger word for a voice recognition service. The verification executor (182) can use the verification engine described above, and can use at least one verification engine.

[0176] The verification result judger (183) can collect verification results from each verification engine and derive a verification result for one starting word.

[0177] The verification result transmitter (184) stores the verification result obtained from the verification result judger (183) in a storage such as a memory, and can transmit the verification result to another service, app, program within the display device (100), another service supported by the display device (100), or a device related thereto.

[0178]

[0179] FIG. 9 illustrates a block diagram of a system for recognition and verification of a starting word for a voice recognition service according to the present invention.

[0180] As described above, the verification engine is described as being embedded in the display device (100), the controller (180) of the display device (100), or another component of the display device (100).

[0181] However, the verification engine may also exist outside the display device (100).

[0182] As an example, as illustrated in FIG. 9, a verification engine may be embedded in a mobile verification server (300). The mobile verification server (300) of FIG. 9 is illustrated as being the same as the server (300) of FIGS. 5 and 6, but may be another server or device.

[0183] The cases in which the verification engine of the mobile verification server (300) is used can be divided as follows.

[0184] 1) When only the verification engine of the mobile verification server (300) is used

[0185] The display device (100) may receive the activation word verification engine information from the server (300) (S551, S552, S651, S652 of FIGS. 5 and 6), and may consider a case where the verification engine of the server (300) is directed by the activation word verification engine information. In this case, the display device (100) may transmit the user's voice (speech) to the server (300).

[0186] The server (300) that receives the user's voice can execute a voice verification engine to perform voice verification on the user's voice.

[0187] The display device (100) can control the verification engines of the mobile verification server (300) to perform verification in parallel when there are multiple verification engines.

[0188] The server (300) can obtain the verification result and transmit it back to the display device (100). After receiving the verification results of all verification engines from the server (300), the display device (100) can determine the final verification result (i.e., whether it is a start word) based on the verification results.

[0189]

[0190] 2) When the verification engine of the mobile verification server (300) is used together with the verification engine of the display device (100)

[0191] This is largely similar to the embodiment 1) above. This is a case where the verification engine information received by the display device (100) includes not only the verification engine of the display device (100) but also the verification engine of the server (300). Even in this case, the display device (100) can transmit the user's voice (speech) to the server (300). In addition, the display device (100) can control its own verification engine and the verification engine of the verification server (300) to perform verification in parallel.

[0192] The server (300) that receives the user's voice can execute a voice verification engine to perform voice verification on the user's voice. The server (300) can obtain the verification result and transmit it back to the display device (100).

[0193] The display device (100) can determine the final verification result based on the verification result received from the server (300) and the verification result obtained by executing its own verification engine.

[0194] Meanwhile, the verification engine of the server (300) used in case 2) may be different from the verification engine embedded in the display device (100).

[0195] When using the verification engine of the server (300), as in 1) or 2), there is an advantage in that a verification algorithm can be applied for a more flexible verification engine. The server (300) collects verification results from the display device and also collects verification results from direct activation word verification, and can optimize or update the verification algorithm using methods such as machine learning.

[0196] In this way, the display device (100) and the server (300) can form a system to perform recognition and verification of start words. The description made with reference to FIGS. 3 to 7, which is not described with reference to FIG. 9, can also be applied to the system of FIG. 9.

[0197]

[0198] In addition, as another aspect of the present invention, the operation of the proposal or invention described above may be implemented, performed or executed by a “computer” (a comprehensive concept including a system on chip (SoC) or a (micro) processor, etc.), or may be provided as a code or a computer-readable storage medium storing or including the code or a computer program product, and the scope of the present invention may be extended to the code or the computer-readable storage medium storing or including the code or the computer program product.

[0199]

[0200] The detailed description of the preferred embodiments of the present invention disclosed above has been provided to enable those skilled in the art to implement and practice the present invention. While the above description has been made with reference to preferred embodiments of the present invention, those skilled in the art will appreciate that various modifications and variations of the present invention, as defined by the following claims, are possible. Accordingly, the present invention is not intended to be limited to the embodiments disclosed herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A display device that provides voice recognition function, A sensor for detecting the user's voice; and A controller for verifying whether the user's voice is a command for activating a voice recognition function, The above controller The user's voice is verified by running multiple verification engines in parallel, and the result of the verification engine that has completed verification of the user's voice first is output first. A display device that determines whether the user's voice is a command for activating the voice recognition function or not based on the results of the plurality of verification engines.

2. A display device according to claim 1, wherein the controller transmits a request for a verification engine to a server and receives information about the plurality of verification engines in response to the request.

3. In the first paragraph, the controller, A display device that obtains the results by applying different weights to the results of the plurality of verification engines.

4. In paragraph 1, A display device wherein the plurality of verification engines are determined based on the hardware or software version of the display device.

5. In the first paragraph, the controller, Transmit the user's voice to the server, The verification engine of the above server receives the result of verifying the user's voice, A display device, wherein the result from the server is used to verify whether the user's voice is a command to activate the voice recognition function.

6. In paragraph 5, The verification engine of the above server is different from the verification engine embedded in the display device.

7. In paragraph 1, The display device wherein the plurality of verification engines are different from the verification engines for verification of startup words supported by the latest version of the operating system software of the display device.

8. A system including a display device and a startup verification server, The display device includes a sensor for detecting a user's voice; and a controller for verifying whether the user's voice is a command for activating a voice recognition function. The above-mentioned activation word verification server includes a controller for receiving the user's voice from the display device and verifying whether the user's voice is a command for activating a voice recognition function. The controller of the above display device Controlling the plurality of verification engines of the display device or the mobile phone verification server to run in parallel to verify the user's voice, A system that determines whether the user's voice is a command for activating the voice recognition function or not based on the results of the verification engine embedded in the display device and the results of the verification engine embedded in the activation word verification server.

9. A method for a voice recognition function, wherein the method is performed by a display device including a sensor for detecting a user's voice; and a controller for verifying whether the user's voice is a command for activating a voice recognition function. A step of recognizing the user's voice; A step of initiating a verification process for the user's voice by running multiple verification engines in parallel; and A method comprising a step of first outputting the result of a verification engine that has first completed verification of the user's voice, and determining whether the user's voice is a command for activating the voice recognition function or not based on the results of the plurality of verification engines.

10. In paragraph 9, A step of sending a request to the verification engine to the server; and A method comprising receiving information about the plurality of verification engines in response to the request.

11. A method comprising a step of obtaining the results by applying different weights to the results of the plurality of verification engines in the 9th paragraph.

12. In the 9th paragraph, the method wherein the plurality of verification engines are determined based on the hardware and software versions of the display device.

13. In paragraph 9, a step of transmitting the user's voice to the server; and A step of receiving a result of verifying the user's voice from the verification engine of the server, A method in which the result from the above server is used to verify whether the user's voice is a command for activating a voice recognition function.

14. In the 13th paragraph, the verification engine of the server is different from the verification engine embedded in the display device.

15. In paragraph 9, A method wherein the plurality of verification engines are different from the verification engines for verification of startup words supported by the latest version of the operating system software of the display device.

Images