Display apparatus including grid non-touch sensor module and method for determining position using the same

Abstract

The present invention relates to a display device and a positioning method comprising a grid-type non-touch sensor module, and more specifically, to a display device and a positioning method comprising a grid-type non-touch sensor module capable of identifying a user interface area, extracting reflective light sensors corresponding to the identified user interface area, and then determining a finger position using a light reception signal output by emitting light from the extracted reflective light sensors.

Description

Technology Field

[0001] The present invention relates to a display device including a grid-type non-touch sensor module and a positioning method. Background Technology

[0002] In the event of an infectious disease outbreak, the disease can spread through objects shared by multiple people. For example, if an infected person places an order through a kiosk equipped with a touchscreen, the touchscreen can become contaminated with pathogens, and the infectious disease can spread through the contaminated touchscreen.

[0003] To solve this problem, Korean Patent No. 2372045 presents a contactless air-touch based kiosk, comprising the steps of: setting the tilt of a plurality of air sensors that detect air touch inputs; detecting the air touch inputs based on a sensing area formed by infrared rays output from the set plurality of air sensors; determining the validity of the detected air touch inputs; if the air touch inputs are determined to be valid inputs, converting the valid touch inputs, which are the air touch inputs determined to be valid inputs, into communication protocol signals of a kiosk control system; and transmitting the converted communication protocol signals to a computing device of the kiosk control system.

[0004] However, this method has a problem in that the air sensor (AS2) that detects the position of the finger prior to clicking and its detection area (SA2) form a horizontal plane horizontal to the kiosk screen, so in order to separate the detection area by a certain distance from the touchscreen, the light-emitting and light-receiving sensors must be installed protruding by a certain distance in a vertical direction from the touchscreen. The problem to be solved

[0005] The problem to be solved by the present invention is to provide a new grid-type non-touch sensor module.

[0006] Another problem to be solved by the present invention is to provide a display device including a novel grid-type non-touch sensor module.

[0007] Another problem to be solved by the present invention is to provide a positioning method in a display device including a novel grid-type non-touch sensor module. means of solving the problem

[0008] A display device including a grid-type non-touch sensor module according to an embodiment of the present invention for achieving the purpose described above may be configured to include: a touchscreen; a grid-type non-touch sensor module comprising a plurality of reflective light sensors arranged in a grid structure below the touchscreen, which irradiate light into a detection area spaced apart from the touchscreen at a predetermined interval and receive reflected light from an object located in the detection area and output it as a light reception signal; and a control unit that identifies a user interface area in display data output on the touchscreen, extracts reflective light sensors corresponding to the identified user interface area, and then determines a finger position using a light reception signal output by emitting light from the extracted reflective light sensors.

[0009] At this time, the control unit can determine a unit grid unit surrounding the finger using the output light reception signal, and determine the finger position using the intensity of the light reception signal output from the reflective light sensors forming the unit grid unit.

[0010] At this time, the control unit can check the user interface mode and determine the unit grid unit by considering the checked user interface mode.

[0011] In addition, the control unit can determine the unit grid unit by considering the detection area.

[0012] In addition, the control unit can determine the finger position by using the light reception signal output by sequentially emitting light from the extracted reflective light sensors one by one or in a row.

[0013] A method for determining a position in a display device comprising a grid-type non-touch sensor module, which is arranged in a grid structure at the bottom of a touchscreen according to an embodiment of the present invention for achieving the purpose described above, and which comprises a plurality of reflective light sensors that irradiate light to a sensing area spaced apart from the touchscreen at a predetermined interval and receive reflected light from an object located in the sensing area and output it as a light reception signal, wherein the display device may comprise the steps of: identifying a user interface area in display data output on the touchscreen; extracting reflective light sensors corresponding to the identified user interface area; and determining a finger position using a light reception signal output by emitting light from the extracted reflective light sensors.

[0014] At this time, the step of determining the finger position may include: a step of determining a unit grid unit surrounding the finger using the output light reception signal; and a step of determining the finger position using the intensity of the light reception signal output from the reflective light sensors forming the unit grid unit.

[0015] At this time, the step of determining the unit grid unit may determine the unit grid unit corresponding to the confirmed user interface mode or detection area after confirming the user interface mode or detection area. Effects of the invention

[0016] According to the present invention, a display device including a novel grid-type non-touch sensor module is provided.

[0017] In particular, a display device including a novel grid-type non-touch sensor module according to the present invention has the advantage of being able to use power more efficiently by selectively emitting light only from the reflective light sensor where a finger is expected to be positioned, taking into account the user interface. Brief explanation of the drawing

[0018] FIG. 1 is a diagram showing the configuration of a grid-type non-touch sensor module according to an embodiment of the present invention. FIG. 2 is a diagram showing the grid arrangement of reflective optical sensors of a grid-type non-touch sensor module according to an embodiment of the present invention. FIG. 3 is a drawing for explaining a reflective optical sensor used in a grid-type non-touch sensor module according to an embodiment of the present invention. FIGS. 4 and 5 are drawings showing the process of reflected light being input to a grid-type non-touch sensor module according to an embodiment of the present invention. FIG. 6 is a diagram showing the detailed operation of a grid-type non-touch sensor module according to an embodiment of the present invention. FIG. 7 is a diagram illustrating the process of determining a unit grid unit surrounding a finger in a grid-type non-touch sensor module according to an embodiment of the present invention. FIG. 8 is a diagram illustrating the process of extracting reflective optical sensors by considering a user interface area according to an embodiment of the present invention. FIG. 9 is a diagram illustrating the process of determining the position of a finger within a unit grid unit in a grid-type non-touch sensor module according to an embodiment of the present invention. FIGS. 10 and 11 are drawings for explaining the process of determining a unit grid unit by considering a user interface mode or a detection area according to an embodiment of the present invention. Specific details for implementing the invention

[0019] In order to clarify the features and advantages of the means for solving the problem of the present invention, the present invention will be described in more detail with reference to specific embodiments of the present invention illustrated in the attached drawings.

[0020] However, detailed descriptions of known functions or configurations that may obscure the essence of the invention are omitted in the following description and the attached drawings. Additionally, it should be noted that identical components throughout the drawings are indicated by the same reference numerals whenever possible.

[0021] Terms and words used in the following description and drawings should not be interpreted as being limited to their ordinary or dictionary meanings, but should be interpreted in a meaning and concept consistent with the technical spirit of the invention, based on the principle that the inventor can appropriately define the concept of terms to best describe his invention. Accordingly, the embodiments described in this specification and the configurations illustrated in the drawings are merely the most preferred embodiments of the invention and do not represent all aspects of the technical spirit of the invention; therefore, it should be understood that various equivalents and modifications capable of replacing them may exist at the time of filing this application.

[0022] The use of terms such as 'first, second, third' preceding the components mentioned in the following description is intended solely to avoid confusion regarding the components being referred to, and is unrelated to the order, importance, or master-subordinate relationship between the components. For example, an invention including only the second component without the first component can also be implemented.

[0023] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0024] First, a display device including a grid-type non-touch sensor module according to an embodiment of the present invention will be described with reference to FIGS. 1 to 6.

[0025] A display device (10) according to the present invention comprises: a touchscreen (100); a grid-type non-touch sensor module (200) comprising a plurality of reflective light sensors (210) arranged in a grid structure below the touchscreen (100), which irradiate light into a detection area spaced apart from the touchscreen (100) at a predetermined interval and receive reflected light from an object located in the detection area and output it as a light reception signal; and a control unit (300) which identifies a user interface area in display data output on the touchscreen (100), extracts reflective light sensors (210) corresponding to the identified user interface area, and then determines a finger position using a light reception signal output by emitting light from the extracted reflective light sensors (210).

[0026] The above touchscreen (100) is configured to detect the location where a finger is touched through the interaction between the finger and the screen when a user touches the screen with a finger, and uses a touchscreen equipped with a pressure-sensitive sensor (not shown).

[0027] As illustrated in FIG. 2, reflective light sensors (210) located at the bottom of the touchscreen (100) are arranged at intersection points where horizontal and vertical lines intersect to form a grid structure. Since the horizontal and vertical lines are spaced equally, the grid structure is formed by square unit grid units (U).

[0028] As illustrated in FIG. 3, the reflective light sensor (210) of the present invention is an integrated reflective light sensor in which a light-emitting element and a light-receiving element are formed as a single unit through a single mold, and comprises a substrate (211), a light-emitting chip (212) mounted on the substrate to emit infrared light, a light-receiving chip (213) mounted side by side, a molding (214) surrounding the light-emitting chip (212) and the light-receiving chip (213), and a lens (215) that refracts and / or concentrates the light emitted or received for inclined light emission and reception.

[0029] The molding (214) of the reflective light sensor (210) is made of black epoxy resin to block extraneous light, and the light-emitting chip (212) is made of a light-emitting diode chip, and an infrared light-emitting diode chip that emits infrared light is used so that it can be separated from ordinary visible light. The light-receiving chip (213) is a photodiode that outputs a microcurrent as an electrical signal when infrared light is incident.

[0030] The above lens (215) can be formed in various shapes, and for example, a hemispherical lens may be used, but is not limited thereto. A light-blocking barrier (216) is formed between the light-emitting chip (212) and the light-receiving chip (213) to prevent direct crosstalk.

[0031] The light emitted from the light-emitting chip (212) diffuses and reaches the lens (215), and is refracted at the lens surface and concentrated with a directional angle within a predetermined range and emitted. The light reflected from the finger (f) is refracted at the lens surface and concentrated.

[0032] As illustrated in FIG. 4, reflective light sensors (210) are positioned at the bottom of the touchscreen (100) and emit light in a vertical direction passing through the touchscreen (100). The reflective light sensors (210) emit light in a diffused manner with a predetermined directional angle. The directional angle range refers to the range in which light is reflected and re-entered into the reflective light sensors (210) when an object is present within the range. Each reflective light sensor (210) is connected to a control unit and emits light in a predetermined sequence of emission from the control unit, and the input reflected light signal is transmitted to the control unit (300).

[0033] As illustrated in FIG. 5, when power is applied to the reflective light sensors (210) in a predetermined order by the control unit (300), if there is no finger in front, the emitted light passes through the detection area and no reflected light is incident, and if there is a finger in front, the reflected light reflected from the finger is introduced. The reflected light input to the reflective light sensor (210) undergoes a photo-current conversion process and is input to the control unit (300). The photodiode receiving chip mounted on the reflective light sensor (210) outputs a microcurrent analogously in proportion to the amount of light received. The output microcurrent is transmitted to the control unit (300) after undergoing conversion into a voltage signal, amplification of the converted signal, and conversion into a digital signal.

[0034] As illustrated in FIG. 6, the control unit (300) includes an analog-to-digital converter (AD CONVERT), a filter and amplifier (OPAMP), a multiplexer (MUX RX), a multiplexer output (MUX TX), a central processing unit (MCU), a USB converter, a buzzer, and a DC input.

[0035] The central processing unit (MCU) of the control unit (300) receives a synchronization signal from the reference generator and causes the IR LEDs to light up in sequence by the multiplexing transmission control circuit (MUX TX) in the grid-type non-touch sensor module (200).

[0036] In the grid-type non-touch sensor module (200), IR signals reflected from a finger are received through a receiving PHOTO TR ARRAY and are received one by one by a MUX RX signal. The IR (Infra Red) light signal is received by the light source reflected from the reflected infrared light through the Photo Tr, and the Photo Tr operates accordingly, so that the voltage between the collector and the emitter decreases differently depending on the intensity of the received light source. The received data is transmitted to an AD converter circuit (ADC) through a switcher. The AD converter (ADC) converts the received analog signal into a digital signal. The reflected light data received from the AD converter is aligned into a data frame and transmitted to a PC via USB.

[0037] Now, the process for determining the finger position according to an embodiment of the present invention will be explained with reference to FIGS. 7 to 10.

[0038] First, as illustrated in FIG. 7, in order to confirm the position of the finger (f), the control unit (300) can emit light from a plurality of reflective light sensors (210) arranged in a grid structure, starting from the first row in a row-by-row manner. Accordingly, there is no signal in the first and second rows, a signal is detected in the third and fourth rows adjacent to the finger (f), and no signal is detected from the fifth row to the ninth row. Additionally, in the third and fourth rows, a signal is generated only in the sixth and seventh columns. Therefore, the control unit (300) can confirm that the finger is located in a unit matrix composed of the light elements (210) in the third and fourth rows and the sixth and seventh columns.

[0039] However, in this method, there is a problem in that all reflective light sensors (210) included in the grid-type non-touch sensor module (200) must be sequentially illuminated to detect a finger. Accordingly, in the present invention, instead of illuminating all reflective light sensors (210), the user interface area is checked from the display data output from the touchscreen (100), the reflective light sensor (210) where the finger is expected to be located is extracted, and the extracted reflective light sensor (210) is illuminated to determine the finger position.

[0040] For example, referring to FIG. 8, the display data output through the touchscreen (100) of the display device (10) can be divided into an area where information is output (510) and an area that supports user input (520). At this time, the area that supports user input (520) may display a number of buttons, and the user can click the corresponding buttons in a non-touch manner. At this time, the control unit (300) of the present invention does not emit light from all reflective light sensors (210), but rather identifies the area that supports user input (520), which is the area where the user is expected to perform non-touch operations, and extracts and emits light only from the reflective light sensors (210) located at the bottom of that area.

[0041] In addition, the control unit (300) of the present invention can determine a unit grid unit surrounding the finger using an output light signal, and determine the final finger position according to the determined unit grid unit.

[0042] That is, as illustrated in Fig. 9, if the finger is positioned above a unit grid unit composed of optical elements 36, 37, 46, and 47, the intensity of light input to the optical elements forming the unit grid unit varies depending on the relative position with respect to the finger. Therefore, the difference in received intensity input to the optical elements forming the unit grid unit is compared to quantify the degree of proximity, and the difference in signal intensity is calculated based on this value to locate the precise position. The calculation method calculates the ratio by calculating the difference in intensity between the upper and lower sensors, and then calculates the signal intensity of the left and right sensors to locate the touch point. For example, if there is a signal intensity from 0 to 100, when X1 is recognized as 30 and X2 as 70, the difference between the two signals X = X2 - X1, and the calculated value is 40. Therefore, the intensity of the nearest touch point is 70 (distance measurement), and if calculated as approximately Xp (touch point) = ((100 - 40) / 100) * 100 with the point slightly offset to the right, it can be extracted as a location offset approximately 60% to the right. A more accurate calculation value can be obtained according to a formula established based on the relationship between intensity and distance.

[0043] Additionally, when clicking or selecting, the position of the finger momentarily moves down toward the touchscreen and then back up, causing the amount of light incident on the 36, 37, 46, and 47 light elements (210) forming the unit grid unit to change in a way that momentarily increases and then decreases. By confirming this change, it is determined that a click or selection action has been performed. Accordingly, the position and movement of the finger are determined.

[0044] In addition, the unit grid unit of the present invention can be set differentially considering the user interface area or the detection area.

[0045] Referring again to FIG. 7, if unit grid unit A is a basic unit grid unit set to determine the position of a general user's finger, the control unit (300) can determine the unit grid unit by considering the user interface area, the detection area, etc., and can also set a unit grid unit that includes more reflective light sensors than the basic unit grid unit, such as unit grid unit B.

[0046] For example, as illustrated in FIG. 10, when the user interface mode is a magnifying glass mode for an elderly user, the control unit (300) can determine a more accurate finger position even if hand tremors occur by setting the unit grid unit to include more reflective light sensors (210), such as the unit grid unit B of FIG. 7.

[0047] Additionally, the control unit (300) can set the detection area as A1, A2, and A3 as shown in FIG. 11, and in the case of A3, the intensity of the reflected light source is weaker and the reflected area is wider compared to A1. Accordingly, if the detection area is set to A3, the control unit (300) can determine a more accurate finger position by setting the unit grid unit to include more reflective light sensors (210) compared to A1.

[0048] The display device (10) including a grid-type non-touch sensor module according to an embodiment of the present invention has been described above.

[0049] The position determination method in a display device (10) including a grid-type non-touch sensor module of the present invention may be provided in the form of a computer-readable medium suitable for storing computer program instructions and data.

[0050] In particular, the computer program of the present invention may perform steps such as identifying a user interface area in display data output to a touchscreen (100) by a display device (10), extracting reflective light sensors (210) corresponding to the identified user interface area, and determining a finger position using a light reception signal output by emitting light from the extracted reflective light sensors (210).

[0051] Computer-readable media suitable for storing such computer program instructions and data include, for example, recording media such as magnetic media (e.g., hard disks, floppy disks, and magnetic tapes), optical recording media (e.g., CD-ROM, Digital Video Disk), and floptical disks (e.g., magneto-optical media), and semiconductor memories such as ROM (Read Only Memory), RAM (Random Access Memory), flash memory, EPROM (Erasable Programmable ROM), and EEPROM (Electrically Erasable Programmable ROM). Processors and memory may be supplemented by or integrated with special-purpose logic circuits.

[0052] Furthermore, computer-readable recording media may be distributed across networked computer systems, allowing computer-readable code to be stored and executed in a distributed manner. Additionally, the functional program for implementing the present invention, along with related code and code segments, may be easily inferred or modified by programmers skilled in the art to which the present invention pertains, taking into account the system environment of the computer that reads the recording media to execute the program.

[0053] In addition, a computer program recorded on a computer-readable recording medium as described above includes instructions that perform the functions described above, and can execute the aforementioned functions by being distributed and circulated through the recording medium, read, installed, and executed on a specific device or specific computer.

[0054] The present invention has been described with reference to embodiments illustrated in the drawings, but this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the present invention should be determined by the technical concept of the claims. Explanation of the symbols

[0055] 10: Display device 100: Touchscreen 210: Reflective optical sensor 200: Grid-type non-touch sensor module 300: Control unit

Claims

Claim 1 A grid-type non-touch sensor module comprising a touchscreen; a plurality of reflective light sensors arranged in a grid structure below the touchscreen, which irradiate light into a detection area spaced apart from the touchscreen at a predetermined interval and receive reflected light reflected from an object located in the detection area to output a light reception signal; and a method for identifying a user interface area in display data output on the touchscreen, extracting reflective light sensors corresponding to the identified user interface area, and then using the light reception signal output by causing the extracted reflective light sensors to emit light. A control unit that determines a unit grid unit surrounding a finger, and then determines a finger position using the intensity of a light reception signal output from reflective light sensors forming the determined unit grid unit; wherein the control unit checks a user interface mode and a set detection area, and if the confirmed user interface mode is a magnifying mode for an elderly user, determines the unit grid unit to include more reflective light sensors than a preset basic unit grid unit, and if the confirmed detection area is set to any one of A1, A2, or A3, determines the unit grid unit to include a number of reflective light sensors corresponding to the set detection area. A display device including a grid-type non-touch sensor module. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 A display device comprising a grid-type non-touch sensor module, wherein, in claim 1, the control unit determines the finger position using a light reception signal output by sequentially emitting light from the extracted reflective light sensors one by one or in a row. Claim 6 A method for determining position in a display device comprising a grid-type non-touch sensor module composed of a plurality of reflective light sensors arranged in a grid structure at the bottom of a touchscreen, which irradiate light into a sensing area spaced apart from the touchscreen at a predetermined interval and receive reflected light from an object located in the sensing area and output it as a light reception signal, wherein the display device comprises the steps of: identifying a user interface area in display data output on the touchscreen; extracting reflective light sensors corresponding to the identified user interface area; determining a finger position using a light reception signal output by emitting light from the extracted reflective light sensors; and determining a unit grid unit surrounding the finger using a light reception signal output by emitting light from the extracted reflective light sensors. A method for determining a position, comprising: a step of determining a finger position using the intensity of a light reception signal output from reflective light sensors forming the determined unit grid unit; wherein the step of determining the unit grid unit involves verifying a user interface mode and a set detection area, and if the verified user interface mode is a magnifying mode for an elderly user, determining the unit grid unit to include more reflective light sensors than a preset basic unit grid unit, and if the verified detection area is set to any one of A1, A2, or A3, determining the unit grid unit to include a number of reflective light sensors corresponding to the set detection area. Claim 7 delete Claim 8 delete

Images