Operation panel device equipped with machine learning-based position estimation unit
The operation panel device uses a two-dimensional array of transparent antennas and machine learning to enhance the accuracy of object position estimation on a display screen, addressing the poor accuracy of existing sensing screen devices.
Patent Information
- Application Number
- JP2024542916
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-09-13
- Publication Date
- 2024-11-11
- Estimated Expiration
- 2042-09-13
AI Technical Summary
Existing sensing screen devices have poor accuracy in estimating the position of a contact object due to the reliance on high-power reflected signals from second antenna units.
An operation panel device with a two-dimensional array of transparent antennas that perform machine learning using reception information from multiple antennas, generating teacher information to accurately estimate the position of an object on a display screen.
The device achieves high-accuracy position estimation of objects on a display screen without physical contact, improving estimation precision and adaptability to environmental changes.
Smart Images

Figure 0007580679000001 
Figure 0007580679000002 
Figure 0007580679000003
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an operation panel device having a function of estimating the position of an object on a display screen of a display panel. [Background technology]
[0002] In recent years, many electronic devices have adopted touch-type operation panels as their user interfaces (hereinafter referred to as UI). Furthermore, Patent Document 1 proposes a sensory screen device that is a non-contact operation panel as a UI. The sensing screen device disclosed in Patent Document 1 includes a transparent antenna layer arranged directly above the display screen of the sensing screen, which includes a first antenna unit that transmits a sensing signal and a plurality of second antenna units that receive reflected signals of the sensing signal, and when a reflected signal of the sensing signal is received, the coordinates of the contact object on the surface on which the sensing screen is located are determined according to the power of the received reflected signal and the coordinates of the second antenna unit that received the reflected signal on the surface. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Special Publication No. 2017-527146 Summary of the Invention [Problem to be solved by the invention]
[0004] The sensing screen device disclosed in Patent Document 1 estimates the position of the contact object using the second antenna unit that receives a reflected signal with a high power among multiple second antenna units that receive reflected signals, and therefore has a problem in that the accuracy of estimating the position of the contact object is poor.
[0005] The present disclosure is devised to solve the above-mentioned problems, and has an object to provide an operation panel device that can estimate the position of an object on the display screen of a display panel with high accuracy. [Means for solving the problem]
[0006] The operation panel device according to the present disclosure includes a display panel having a display screen, a plurality of transparent antennas arranged two-dimensionally on the display screen of the display panel, a received information generation unit that generates received information corresponding to the plurality of transparent antennas linked to transparent antennas that receive reflected waves based on received signals from the plurality of transparent antennas that receive reflected waves from an object present on the display screen of the display panel, a storage unit that stores teacher information corresponding to the plurality of transparent antennas linked to position information on the display screen of the display panel, and an object position estimation unit that performs machine learning using the teacher information stored in the storage unit on the received information generated by the received information generation unit to estimate the position of the object on the display screen of the display panel. a teacher information generating unit that generates teacher information to be stored in a storage unit, the teacher information generating unit generating teacher information in which position information on the display screen of the display panel according to the instruction image is linked to reception information corresponding to a transparent antenna that receives a reflected wave based on reception signals from a plurality of transparent antennas that receive a reflected wave from an object that exists along the instruction image displayed on the display screen of the display panel; and Equipped with. Effect of the Invention
[0007] According to the present disclosure, an object position estimation unit performs machine learning using teacher information stored in a storage unit on the received information corresponding to the multiple transparent antennas generated by the received information generation unit, and estimates the position of the object on the display screen of the display panel, thereby enabling the position of the object on the display screen of the display panel to be estimated with high accuracy. [Brief description of the drawings]
[0008] [Figure 1] 1 is a block diagram showing an operation panel device according to a first embodiment. [Diagram 2] 2 is a schematic diagram showing a surface of a panel in which a transparent antenna array is embedded in the operation panel device in accordance with the first embodiment. FIG. [Diagram 3] 4 is an explanatory diagram showing up-chirp signals Tx(1) to Tx(N) generated by a signal generator in the operation panel device according to the first embodiment. FIG. [Figure 4]2 is a schematic front view showing the relationship between a finger and an instruction image displayed on a display screen of a panel in the operation panel device in accordance with the first embodiment. FIG. [Diagram 5] 2 is a hardware configuration diagram showing a first example of an object position estimating section 40 in the operation panel device according to the first embodiment, which is realized by dedicated hardware. FIG. [Figure 6] 11 is a hardware configuration diagram of a computer realized by software, firmware, or the like, showing a second example of the target object position estimating section 40 in the operation panel device according to the first embodiment. FIG. [Figure 7] 5 is a flowchart showing a procedure for a position estimating operation for performing a process of estimating a finger position in the operation panel device in accordance with the first embodiment. [Figure 8] 10 is a flowchart showing a procedure of a calibration execution operation for generating teacher information in the operation panel device in accordance with the first embodiment. [Figure 9] 11 is a schematic front view showing the relationship between a finger and an instruction image displayed on a display screen of a panel in an operation panel device in accordance with embodiment 2. FIG. [Figure 10] FIG. 11 is a block diagram showing an operation panel device according to a third embodiment. [Figure 11] FIG. 11 is a block diagram showing an operation panel device according to a fourth embodiment. [Figure 12] FIG. 13 is a block diagram showing an operation panel device according to a fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Embodiment 1 An operation panel device according to a first embodiment will be described with reference to FIGS. 1 to 8. FIG. The operation panel device of embodiment 1 is a non-contact operation panel with a calibration function, and includes a display panel (hereinafter abbreviated as panel) 10, a transparent antenna array 20, a transmission / reception circuit unit 30, an object position estimation unit 40, and a control unit 50. In other words, the operation panel device of embodiment 1 is a user interface (UI) that enables screen operation by bringing an object such as a finger or a stylus (hereinafter, a finger will be used as a representative object) close to the panel without physically touching the panel 10.
[0010] The operation panel device of embodiment 1 has a function of estimating the position of the finger on the display screen of the panel 10 by performing machine learning using teacher information on received information based on received signals from multiple transparent antennas that constitute the transparent antenna array 20 that receive reflected waves from the finger, i.e., a position estimation function. Furthermore, the operation panel device has a function of acquiring a plurality of pieces of teacher information each associated with a respective piece of reception information based on reception signals from a plurality of transparent antennas, that is, a function of performing calibration.
[0011] The panel 10 has a display screen. The transparent antenna array 20 is a set of multiple transparent antennas 2 embedded in a two-dimensional space inside the panel 10 in order to improve the accuracy of finger position estimation by MIMO (Multiple-Input Multiple-Output). 11 ~2 mn It consists of:
[0012] Multiple transparent antennas 2 11 ~2 mn 2, are two-dimensionally arranged in m rows and n columns on the entire surface of the display screen of panel 10. m and n are natural numbers, and at least one of them is a natural number of 2 or more. For convenience of explanation, transparent antenna 2 11 The transparent antenna 21 is the first transparent antenna 21, the transparent antenna 212 is the second transparent antenna 22, and the transparent antenna 2 mn Nth transparent antenna 2 N In other words, the transparent antenna array 20 is made up of a first transparent antenna 21 to an N-th transparent antenna 22. N The antenna is composed of N transparent antennas, where N is a natural number greater than or equal to 2. In addition, the first transparent antenna 21 to the Nth transparent antenna 2 N If there is no need to distinguish between the two, use the transparent antenna 2. k The following explanation assumes that k=1 to N.
[0013] Transparent Antenna 2 k is a transmitting and receiving antenna that serves as both a transmitting antenna and a receiving antenna. In addition, transparent antenna 2 k In the case of the 1st embodiment, the transmitting antennas and the receiving antennas may be separate. Also, when the number of receiving antennas is N, the number of transmitting antennas may be less than the number of receiving antennas, for example, N / 2. Transparent Antenna 2 k When it functions as a transmitting antenna, it radiates a transmission wave consisting of an electric wave onto the display screen of the panel 10, that is, into the space in which the finger is present. The transmission wave is a high frequency wave generated by a transmission signal in a high frequency band such as L band, S band, C band, or X band.
[0014] Transparent Antenna 2 k When it functions as a receiving antenna, it receives waves reflected by a finger placed on the display screen of the panel 10 and outputs a reception signal based on the reflected waves. Transparent Antenna 2 k is an antenna made of a transparent conductor such as a metal mesh of fine wiring, an indium tin oxide film, or graphene, making its shape invisible to the naked eye.
[0015] The transmission / reception circuit section 30 includes an input / output switching section 31 , a signal transmission section 32 , and a reception information generation section 33 . The transmitting / receiving circuit unit 30 transmits the first transparent antenna 21 to the Nth transparent antenna 22 every cycle. N Select the transparent antenna 2 by selecting k A high-frequency transmission signal is output to the selected transparent antenna 2. k The first transparent antenna 21 to the Nth transparent antenna 22 receive the reflected wave from the finger on the display screen of the panel 10 for each transmitted wave. NReceives a signal from the target and selects transparent antenna 2 as the transmitting antenna based on the received signal. k The first transparent antenna 21 to the Nth transparent antenna 2 associated with the transparent antenna that received the reflected wave are N Generate and output reception information corresponding to the received signal.
[0016] The input / output switching unit 31 transmits the transparent antenna 2 selected by the signal transmitting unit 32. k The signal transmitting unit 32 transmits a high-frequency transmission signal to the first transparent antenna 21 through the Nth transparent antenna 2 which receive the reflected wave from the finger. N The received signal from the receiving unit 31 is transmitted to the receiving information generating unit 33. The input / output switching unit 31 is a first transparent antenna 21 to an Nth transparent antenna 2 N In response to the above, the first switching unit 311 to the Nth switching unit 31 N has. The first switching unit 311 to the Nth switching unit 31 N When there is no need to distinguish between the above, the switching unit 31 k The following explanation assumes that k=1 to N.
[0017] Switching unit 31 k is a three-port circulator commonly known in the field of high frequency signals. Switching unit 31 k has an input port, an input / output port, and an output port. Switching unit 31 k The input port of each is connected to the output end of the signal transmitting unit 32, and the input / output port of each is connected to the corresponding transparent antenna 2 k , and the output port is connected to the received information generating unit 33. Switching unit 31 k When transmitting, the input port and the input / output port are connected, and when receiving, the input / output port and the output port are connected.
[0018] Switching unit 31 k Transparent antenna 2 k and the reception information generating unit 33 are placed in a connected state. The switching unit 31 selected by the signal transmitting unit 32 kThe corresponding transparent antenna 2 k and the signal transmitting unit 32 are connected to the transparent antenna 2. k After outputting to Transparent Antenna 2 k and the reception information generating unit 33 are placed in a connected state.
[0019] The signal transmission unit 32 switches the first switching unit 311 to the Nth switching unit 31 N The selected switching unit 31 k A high frequency transmission signal is output to the At this time, the first switching unit 311 to the Nth switching unit 31 N The selection order is the first switching unit 311, the second switching unit 312, . . . , the Nth switching unit 31 N In FIG. 2, the transparent antennas are selected from the left end to the right end of the first row, from the left end to the right end of the second row, and from the left end to the right end of the mth row. The first switching unit 311 to the Nth switching unit 31 N The selection order is the Nth switching unit 31 N , the (N-1)th switching unit 31 N-1 , . . . , the first switching unit 311, etc. may be used, and the switching order is not important.
[0020] The signal transmitting unit 32 includes a signal generator 321 and an output destination selecting unit 322 . The signal generator 321 generates an up-chirp signal or a down-chirp signal according to a Frequency Modulated Continuous Wave (FMCW) method or a Fast-Chirp Modulation (FCM) method, and outputs N transmission signals per transmission cycle. The signal generator 321 repeatedly outputs N transmission signals for each transmission cycle c (c=1, 2, . . . ).
[0021] 3, in the first embodiment, the signal generator 321 generates up-chirp signals Tx1 to TxN whose frequencies change over time for each transmission cycle, up-converts the frequencies of the up-chirp signals Tx1 to TxN to signals in a high frequency band, and outputs the up-converted signals as transmission signals TX1 to TXN to the output destination selection unit 322. Examples of the high frequency band include frequency bands such as L band, S band, C band, and X band. The signal generator 321 generates up-chirp signals Tx1 to TxN in response to a timing signal indicating the time from the control unit 50.
[0022] The number N of transmission signals TX1 to TXN in one transmission cycle is the same as the number N of the first transparent antenna 21 to the Nth transparent antenna 2N, and in embodiment 1, the first transmission signal TX1 to the Nth transmission signal TXN in one transmission cycle correspond to the first transparent antenna 21 to the Nth transparent antenna 2N. The order of generation of the up-chirp signals Tx1 to TxN by the signal generator 321 is the same as the order of selection as transmitting antennas from the first transparent antenna 21 to the Nth transparent antenna 2N, and the first signal Tx1, the second signal Tx2, . . . , the Nth signal TxN Order of It is.
[0023] When there is no need to distinguish between the N transmission signals TX1 to TXN, they will be described as a transmission signal TXk (k=1 to N). The transmission signal TXk may be a signal based on a pulse signal, instead of a signal based on a signal whose frequency changes over time, such as an up-chirp signal or a down-chirp signal.
[0024] The output destination selection unit 322 acquires a transmission signal TXk from the signal generator 321 in synchronization with a timing signal indicating time from the control unit 50, and selects the transparent antenna 2 corresponding to the acquired transmission signal TXk. k The switching unit 31 to which k Select the selected switch 31 k The signal TXk is output to the The output terminal of the output destination selection unit 322 is connected to the first switching unit 311 through the Nth switching unit 31 N Input Connected to the port.
[0025] The received information generating unit 33 receives a reflected wave from a finger present on the display screen of the panel 10, and outputs the reflected wave to the first transparent antenna 21 through the Nth transparent antenna 22. N The received signal is transmitted from the first switching unit 311 to the Nth switching unit 31 N 2, which is selected as the transmitting antenna based on the received signal input via the k and transparent antenna 2 receiving the reflected wave. k The first transparent antenna 21 to the Nth transparent antenna 2 N Generate and output reception information corresponding to the received signal.
[0026] The reception information generating unit 33 receives the first transparent antenna 21 to the Nth transparent antenna 2 N In response to the above, the first switching unit 311 to the Nth switching unit 31 N From the first signal receiving unit 331 corresponding to No. N Signal receiving unit 33 N has. The first signal receiving unit 331 to the Nth signal receiving unit 33 N When there is no need to distinguish between the signal receiving unit 33 and the signal receiving unit 33, k The following explanation assumes that k=1 to N.
[0027] Signal receiving unit 33 k The input terminal of the corresponding switching unit 31 k is connected to the output port of Signal receiving unit 33 k is the corresponding transparent antenna 2 k The received signal is then input to the corresponding switching unit 31. k Receive via. Signal receiving unit 33 khas a downconversion unit that downconverts the high frequency band of the received signal to a frequency in the intermediate frequency band, and an analog / digital conversion unit that A / D converts the received signal, which is an analog signal downconverted by the downconversion unit, into a digital received signal.
[0028] Signal receiving unit 33 k indicates the time from the control unit 50 timing Transparent antenna 2 is receiving a signal and is selected as the transmitting antenna k Information and selected transparent antenna 2 k The timing when the transparent antenna 2 emitted the transmission wave and the corresponding k receives the reflected wave from the finger and the corresponding switching unit 31 k and acquires the timing at which the received signal was received via the corresponding switch unit 31. k The frequency and strength of the received signal are obtained via.
[0029] Selected Transparent Antenna 2 k The time when the first transparent antenna 21 radiates the transmission wave is the time when the Nth transparent antenna 2 N Since the selection order of the signal receiving unit 33 is already determined, k Transparent antenna 2 selected k The time at which the transmission wave is to be emitted is input via the control unit 50. The time when the received signal was received is recorded in the signal receiving unit 33. k The time is acquired based on a signal indicating the time from the control unit 50.
[0030] Signal receiving unit 33 k According to the acquired information, the corresponding switching unit 31 k Based on the received signal, transparent antenna 2 is selected as the transmitting antenna. k and transparent antenna 2 receiving the reflected wave. k The reception information S(t, g, h) is generated as digital data in which the above-mentioned signals are linked to each other, and is output to the object position estimation unit 40. The received information S(t, g, h) is transmitted to the transparent antenna 2. kThe received information S(t, g, h) may include information indicating the frequency and intensity of the received signal obtained from the received information S(t, g, h). The received information S(t, g, h) may include a change in the shape of the spectrum of the received signal.
[0031] In the received information S(t, g, h), t indicates the sampling time with the sampling start time of each receiving antenna being 0. (Receiving) Transparent antenna 2 k The digital data is the reception time of the reflected wave, g is the (transmitting) transparent antenna 2 selected as the transmitting antenna. k The transmitting antenna identification information (e.g. g = 1 to N) is digital data that identifies the transmitting antenna. h is the transparent antenna 2 that received the reflected wave. k The receiving antenna identification information is digital data that identifies the receiving antenna (e.g., h=1 to N).
[0032] Now, assuming that the first transparent antenna 21 is selected as the transmitting antenna, a transmission signal TX1 is output from the signal generator 321, the first switching unit 311 is selected by the output destination selection unit 322, the transmission signal TX1 is input to the first transparent antenna 21, and the first transparent antenna 21 radiates a transmission wave consisting of radio waves onto the display screen of the panel 10, that is, into the space where the finger is present. k-th signal receiving unit 33 k The corresponding transparent antenna 2 receives the reflected wave from the finger. k A switching unit 31 corresponding to a received signal based on a reflected wave from k When it receives via, it generates received information S(t,1,k).
[0033] When the first transparent antenna 21 is selected as the transmitting antenna, the first signal receiving unit 331 No. N Signal receiving unit 33 N Since reception information S is generated for all, reception information generator 33 generates N pieces of reception information S(t, 1, h). In one transmission cycle, the first transparent antenna 21 to the Nth transparent antenna 2 N are selected as transmitting antennas in order, the reception information generating unit 33 generates (N×N) pieces of reception information S(t, g, h) and outputs them to the object position estimating unit 40.
[0034] The object position estimation unit 40 has two functions. The first function is a function of performing machine learning using teacher information on the reception information S(t, g, h) generated by the reception information generating unit 33, in this example, the (N×N) pieces of reception information S(t, g, h) generated in one transmission cycle, and estimating the position of the finger on the display screen of the panel 10. As is generally known, machine learning using teacher information is performed to create a learning model, and the learning model is used to estimate the position of the finger on the display screen of the panel 10.
[0035] The second function is to display an instruction image IP for indicating the presence of a finger on the display screen of the panel 10, and transmit the reflected wave from the finger at a set position along the displayed instruction image IP to the first transparent antenna 21 to the Nth transparent antenna 2. N This is a function to generate teacher information S(t, g, h|x, y), which is calibration result data linked to position information (x, y) on the display screen of panel 10 by instruction image IP, using reception information corresponding to the transparent antenna that received the reflected wave, that is, (N×N) pieces of reception information S(t, g, h) per transmission cycle generated by reception information generating unit 33 as calibration information based on the reception signal from the transparent antenna, i.e., a calibration function. x, y are the x and y coordinates on the display screen of a finger that is located at a set position along the instruction image IP that has received the reflected wave. Note that the vertical coordinate (z coordinate) with respect to the display screen of panel 10 may be added as position information.
[0036] The teacher information S(t, g, h|x, y) is the position information (x, y) on the display screen of the panel 10 by the instruction image IP and the reflected wave of the finger at this position from the first transparent antenna 21 to the Nth transparent antenna 2 N From the first signal receiving unit 331 by each of the received signals from No. N Signal receiving unit 33 NThe information generated as pairs of all received information S(t, g, h) generated by each of them.
[0037] The object position estimation unit 40 includes a received information acquisition unit 41 and a machine learning unit 42 . The reception information acquisition unit 41 receives the first signal from the first signal receiving unit 331. No. N Signal receiving unit 33 N The received information S(t, g, h) generated by each is obtained. The reception information acquisition unit 41 receives the first signal reception unit 331 to the first signal reception unit 33 N The received information generator 33 obtains (N×N) pieces of received information S(t, g, h) per transmission cycle c, and provides the information to the machine learning unit 42.
[0038] The machine learning unit 42 has the above-mentioned first function and second function. The machine learning unit 42 includes a position estimation unit 421 , a teacher information storage unit 422 , and a teacher information generation unit 423 . The position estimation unit 421 performs machine learning on the received information S(t, g, h) acquired by the received information acquisition unit 41, in this example, (N×N) pieces of received information S(t, g, h) per transmission cycle c, using multiple pieces of teacher information S(t, g, h|x, y) linked to multiple pieces of position information (x, y) on the display screen of panel 10 stored in the teacher information storage unit 422, estimates the position (X, Y) of the finger on the display screen of panel 10, and outputs the estimated position of the finger.
[0039] The machine learning by the position estimation unit 421 is machine learning using either a machine learning method of deep learning or Gaussian process regression. Instead of the machine learning method using deep learning and Gaussian process regression, it may be realized by another machine learning framework. When deep learning is used as a machine learning method, the weight parameters connecting each layer correspond to the teaching information.
[0040] For example, in a machine learning framework using deep learning, the training data S(t, g, h|x, y) obtained at all positions (x, y) is used as training data input to train a deep learning model that outputs the true finger position (x, y). All weight parameters in the deep learning model are adjusted by backpropagation or the like so that the deep learning model output is close to the true finger position (x, y). All these weight parameters correspond to training data in the deep learning model.
[0041] When Gaussian process regression is used as a machine learning method, the Gram matrix parameters correspond to the training information. For example, in Gaussian process regression, the degree of approximation between all pairs of teacher information S(t, g, h|x, y) acquired at all positions (x, y) is calculated using a default kernel function, and a Gram matrix with M rows and M columns is generated with the kernel function outputs as matrix elements. Here, M is the number of all learning positions (x, y). The default kernel function may be, for example, a Gaussian kernel. A finger position estimation method using Gaussian process regression is a method of estimating an unknown finger position (X, Y) by performing a matrix operation using this Gram matrix on received information S(t, g, h|X, Y) when a finger is at an unknown position (X, Y), and this Gram matrix corresponds to the teacher information. When using other supervised machine learning frameworks, the parameters of the supervised machine learning model correspond to the training information.
[0042] The teacher information generating unit 423 is controlled by the control unit 50 in cooperation with the panel 10 and the transmission / reception circuit unit 30 during the calibration operation for acquiring a plurality of pieces of teacher information. The teacher information generating unit 423 causes the control unit 50 to display, on the display screen of the panel 10, an instruction image IP for indicating that a finger should be placed thereon. The instruction image IP is an image for instructing the user to slowly trace the leading edge of the slowly moving image with his / her finger.
[0043] As shown in FIG. 4, the instruction image IP is a drawing image that shows a zigzag trajectory that slowly draws a drawing line from the upper left corner to the upper right corner of the display screen of panel 10, steps down, goes back and forth several times from the right edge to the left edge, and ends from the lower right edge to the lower left edge. 4, the leading edge of the drawing line, which is the instruction image IP, moves slowly across the display screen of panel 10, and the part that moves becomes the dark drawing line IPb. The light drawing line IPa represents the drawing line from which the dark drawing line IPb extends. The drawn line, which is the instruction image IP, indicates an instruction that the user slowly traces with his / her finger on the display screen of the panel 10, following the movement of the leading portion. It takes several tens of seconds for the leading part to move over the entire thin drawn line IPa.
[0044] The teacher information generating unit 423 generates a plurality of setting positions P d (d=1 to D), that is, the first set position P1 to the Dth set position P D In each case, (N×N) pieces of reception information S(t, g, h) per transmission cycle c generated by the reception information generator 33 are acquired as calibration information. D is a natural number of 2 or more. That is, the set position P d In each case, the teacher information generating unit 423 selects a transmission signal from the signal transmitting unit 32 and transmits the selected signal to the transparent antenna 2. k Switching unit 31 k Enter through the transparent antenna 2 selected k The transmitting wave is emitted from
[0045] Selected transparent antenna 2 k The first transparent antenna 21 to the Nth transparent antenna 2 receive a reflected wave from the finger in response to a transmitted wave from the N Based on the received signal from the transparent antenna, the teacher information generating unit 423 acquires the received information corresponding to the transparent antenna that received the reflected wave, that is, the received information S(t, g, h) generated by the received information generating unit 33, via the received information acquiring unit 41. The teacher information generating unit 423 slowly draws a zigzag trajectory from the beginning (upper left end) to the end (lower left end) of the leading portion of the drawing line, which is the instruction image IP, and sets the drawing line setting position P d In each of them, the reception information S(t, g, h) generated by the reception information generator 33 is acquired via the reception information acquirer 41.
[0046] Setting position P d In each of them, in one transmission cycle c, the first transparent antenna 21 to the Nth transparent antenna 2 N are selected in order as transmitting antennas, and the teacher information generating unit 423 acquires (N×N) pieces of reception information S(t, g, h) from the reception information generating unit 33 via the reception information acquiring unit 41. Therefore, the finger present on the display screen of the panel 10 follows the leading edge of the drawing line, which is the instruction image IP, until the finger finishes tracing the drawing line. D In all cases, the teacher information generating unit 423 acquires (N×N) pieces of reception information S(t, g, h) from the reception information generating unit 33 via the reception information acquiring unit 41.
[0047] Setting position P d In each of the transparent antennas, the teacher information generating unit 423 generates the teacher information for the first transparent antenna 21 to the Nth transparent antenna 2 N The setting position P is selected as the transmitting antenna in order. d Position information (x d , y d ) and the training information S(t, g, h|x d , y d ) and generate the teacher information S(t, g, h|x d , y d ) is stored in the teacher information storage unit 422.
[0048] That is, the teacher information generating unit 423 generates the position information (x d , y d) (d=1 to D), that is, the position information (x d , y d The teacher information S(t, g, h|x, y) to which the instruction image IP is linked is displayed at positions set over the entire area of the instruction image IP on the display screen of the panel 10, that is, the first setting position P1 to the Dth setting position P D Generated in each. The position information of the leading portion is expressed by the coordinates of the x-axis and y-axis on the display screen of panel 10. Furthermore, the position information of the leading portion may further include the coordinates of the z-axis in the vertical direction to the display screen of panel 10, that is, on the xy plane.
[0049] Next, a specific configuration of the object position estimation unit 40 will be described. As a first example, the reception information acquisition unit 41 and the machine learning unit 42 are realized by dedicated hardware such as a reception information acquisition circuit and a machine learning circuit, respectively, as shown in FIG. The reception information acquisition circuit corresponds to reception information acquisition unit 41, and the machine learning circuit corresponds to machine learning unit .
[0050] The reception information acquisition circuit is an interface circuit that transmits the reception information S(t, g, h) from the reception information generation unit 33 to the machine learning circuit, and a generally known interface circuit can be used. The machine learning circuit includes a position estimation circuit constituting the position estimation unit 421, a memory which is the teacher information storage unit 422, and a teacher information generation circuit constituting the teacher information generation unit 423. Each of the position estimation circuit and the teacher information generation circuit is realized, for example, by a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of these.
[0051] As a second example, the reception information acquisition unit 41 and the machine learning unit 42 are realized by a computer in which a program of software, firmware, or a combination of software and firmware is embedded, as shown in FIG. Figure 6 shows a typical example of a computer. A computer has a CPU (Central Processing It is equipped with a serial communication unit (SRAM), an input interface unit, RAM (Random Access Memory), ROM (Read Only Memory), and an output interface unit. The CPU controls and manages the input interface section, RAM, ROM, and output interface section.
[0052] The CPU, in cooperation with the ROM and RAM, fulfills the functions of the position estimation unit 421 and the teacher information generation unit 423. The ROM stores a position estimation program that executes a position estimation processing procedure to function as a position estimation unit 421, and a teacher information generation program, which is a calibration execution program that executes a processing procedure for generating teacher information to function as a teacher information generation unit 423. ROM is a type of recording medium, such as a DVD (Digital Versatile Disc), a CD (Compact Disc), a HDD (Hard Disk Drive), or a USB memory. The RAM functions as a teacher information storage unit 422, and also plays a role in temporarily storing a program stored in the ROM when the CPU executes the program stored in the ROM.
[0053] The input interface unit corresponds to a reception information acquisition unit 41 that acquires reception information S(t, g, h) from the reception information generation unit 33, and when the computer performs a calibration execution operation, acquires position information (x, y) of the top part on the display screen of the panel 10 based on instruction information, etc. When the computer performs a position estimation operation, the output interface unit outputs the estimated position of the finger, which is the estimation result by the CPU, ROM, and RAM functioning as a position estimation unit 421, and when the computer performs a calibration execution operation, the output interface unit outputs instruction image information, etc., for displaying an instruction image IP on the display screen of the panel 10 by the CPU, ROM, and RAM functioning as a teacher information generation unit 423.
[0054] Although the computer uses a CPU in the above embodiment, it may be a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, or hardware that executes a DSP (Digital Signal Processor) program. Furthermore, without being limited to the first and second examples, some of the components may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, or the like.
[0055] Next, the operation of the operation panel device according to the first embodiment will be described. First, the position estimation operation in one transmission cycle c (c=1) will be described with reference to Fig. 7. Although the transmission cycle c is performed multiple times, the position estimation operation is the same, so only one transmission cycle c (c=1) will be described. In step ST11, transparent antenna 2 is selected as the transmitting antenna. k is initially set to the first transmitting antenna 21 (k=1), and in step ST12, in order to radiate a transmission wave from the first transparent antenna 21, the output destination selection unit 322 selects the first switching unit 311 to output the first transmission signal TX1 from the signal generator 321 to the first transparent antenna 21.
[0056] The first transparent antenna 21 radiates a transmission wave into the space in which the finger is present. The transmission wave radiated from the first transparent antenna 21 is reflected by the finger and is transmitted from the first transparent antenna 21 to the Nth transparent antenna 2 N receives the wave reflected by the finger (step ST13). The first transparent antenna 21 to the Nth transparent antenna 2 N The received signals due to the reflected waves received by the first switching unit 311 to the Nth switching unit 31 N From the first signal receiving unit 331 via each No. N Signal receiving unit 33 N is entered into
[0057] From the first signal receiving unit 331 No. N Signal receiving unit 33 N Each performs reception processing on the input reception signal, and determines that the transparent antenna selected as the transmitting antenna is the first transparent antenna 21, the time when the first transparent antenna 21 radiated the transmission wave, and the corresponding transparent antenna 2 k The sensor generates reception information S(t, 1, h(1 to N)) based on the received reception signal at the time when the sensor receives the reflected wave from the finger (step ST14).
[0058] From the first signal receiving unit 331 No. N Signal receiving unit 33 N When each of them generates the reception information S(t, 1, h(1 to N)) when the first transparent antenna 21 is selected as the transmission antenna, the process proceeds to step ST15, and the transparent antenna 2 selected as the transmission antenna is k to the second transmitting antenna 22 (k=2), and the process returns to step ST12. In step ST12, in order to radiate a transmission wave from the second transmitting antenna 22 (k=2) as a transmitting antenna, the output destination selection unit 322 selects the second switching unit 312 to output the second transmission signal TX2 from the signal generator 321 to the second transparent antenna 22.
[0059] A second transparent antenna 22 radiates the transmitted wave into the space in which the finger is present. The transmission wave radiated from the second transparent antenna 22 is reflected by the finger and transmitted to the first transparent antenna 21 to the Nth transparent antenna 22. N receives the wave reflected by the finger (step ST13). The first transparent antenna 21 to the Nth transparent antenna 2 NThe received signals due to the reflected waves received by the first switching unit 311 to the Nth switching unit 31 N From the first signal receiving unit 331 via each No. N Signal receiving unit 33 N is entered into
[0060] From the first signal receiving unit 331 No. N Signal receiving unit 33 N Each of them performs reception processing on the input reception signal, and determines that the transparent antenna selected as the transmitting antenna is the second transparent antenna 22, the time when the second transparent antenna 22 radiated the transmission wave, and the corresponding transparent antenna 2 k The sensor generates reception information S(t, 2, h(1 to N)) based on the received reception signal at the time when the sensor receives the reflected wave from the finger (step ST14).
[0061] In this way, the Nth transparent antenna 2 is used as a transmitting antenna. N Transparent Antenna 2 until is selected k are selected as transmitting antennas in order, and the loop from step ST12 to step ST15 is repeated. No. N Signal receiving unit 33 N Thus, (N×N) pieces of received information S(t, g, h) are generated.
[0062] In step ST14, the reception information generating unit 33 receives a reflected wave from a finger present on the display screen of the panel 10, and transmits the reflected wave to the first transparent antenna 21 to the Nth transparent antenna 22 that are two-dimensionally arranged on the display screen of the panel 10. N from A transparent antenna 2 receives a reflected wave based on the received signal. k The first transparent antenna 21 to the Nth transparent antenna 2 N This is the step of generating reception information S(t, g, h) corresponding to
[0063] In step ST15, the Nth transparent antenna 2 is used as a transmitting antenna. NIf selected, proceed to step ST16. In step ST16, the reception information acquisition unit 41 acquires the (N×N) pieces of reception information S(t, g, h) from the reception information generation unit 33, and provides the (N×N) pieces of reception information S(t, g, h) to the position estimation unit 421.
[0064] The position estimation unit 421 performs machine learning on the (N×N) pieces of received information S(t, g, h) using multiple pieces of teacher information S(t, g, h|x, y) linked to each of multiple pieces of position information (x, y) on the display screen of panel 10, which are stored in the teacher information storage unit 422, estimates the position (X, Y) of the finger on the display screen of panel 10, and outputs the estimated position of the finger. Subsequent estimation of the finger position is performed using a learning model generated as a result of machine learning.
[0065] Step ST16 is a step in which the position estimation unit 421 in the object position estimation unit 40 performs machine learning on the (N×N) pieces of received information S(t, g, h) generated by the received information generation unit 33 using the teacher information S(t, g, h|x, y) stored in the teacher information storage unit 422, and estimates the position (X, Y) of the finger on the display screen of the panel 10.
[0066] Furthermore, when the reception information acquisition unit 41 and the machine learning unit 42 are realized by a computer, a program for executing the process of step ST16 is stored in a ROM, and the CPU executes the program stored in the ROM. The program stored in the ROM transmits a reflected wave from a finger on the display screen of the panel 10 to a first transparent antenna 21 to an Nth transparent antenna 22 arranged two-dimensionally on the display screen of the panel 10. N The first transparent antenna 21 to the Nth transparent antenna 2 associated with the transparent antenna that received the reflected wave based on the received signal from the Nand a procedure for performing machine learning on the acquired received information S(t, g, h) using teacher information S(t, g, h|x, y) to estimate the position (x, y) of the finger on the display screen of the operation panel device.
[0067] Next, the calibration execution operation in the operation panel device of embodiment 1, that is, the operation of generating teacher information S(t, g, h|x, y), which is calibration result data linked to position information (x, y) on the display screen of panel 10, will be described with reference to FIG. 8. The teacher information generating section 423 is controlled by the control section 50 in cooperation with the panel 10 and the transmission / reception circuit section 30 .
[0068] In step ST21, the teacher information generating section 423 outputs instruction image information for displaying an instruction image IP on the display screen of the panel 10 to the panel 10 via the control section 50. As shown in FIG. 4, the panel 10 displays a thin, zigzag drawing line IPa on the display screen, and the leading edge of the thin drawing line IPa moves slowly along the line until the moving part becomes a thicker drawing line IPb, thereby displaying an instruction image IP instructing the user to slowly trace the line with their finger along the movement of the leading edge of the thicker drawing line IPb (step ST22).
[0069] When the leading edge of the dark drawn line IPb in the instruction image IP reaches the first set position P1, the teacher information generating unit 423 acquires position information (x1, y1) of the leading edge of the instruction image IP at the first set position P1 (step ST23). At the same time, the teacher information generating unit 423 sets the first transparent antenna 21 to the Nth transparent antenna 2 at the first setting position P1. N When each of the first transparent antenna 21 to the Nth transparent antenna 2 is used as a transmitting antenna, N The (N×N) pieces of reception information S(t, g, h) generated by the reception information generating unit 33 based on the reception signal received by the reception information acquiring unit 41 are acquired (step ST24).
[0070] In step ST24, the (N×N) pieces of reception information S(t, g, h) acquired by the teacher information generation unit 423 are the same as the (N×N) pieces of reception information S(t, g, h) generated by the reception information generation unit 33 from steps ST11 to ST15 in the position estimation operation. That is, although detailed description is omitted, at the first setting position P1, the first transparent antenna 21 to the Nth transparent antenna 2 are set as transmitting antennas. N are selected in order, and the loop from step ST12 to step ST15 is repeated, and the first signal receiving unit 331 to No. N Signal receiving unit 33 N The teacher information generating unit 423 acquires the (N×N) pieces of reception information S(t, g, h) generated by:
[0071] In step ST25, the teacher information generation unit 423 generates teacher information S(t, g, h|x1, y1) at the first setting position P1 by pairing the acquired position information (x1, y1) of the first setting position P1 with the acquired (N×N) pieces of received information S(t, g, h) at the first setting position P1. The teacher information generating unit 423 stores the generated teacher information S(t, g, h|x1, y1) at the first setting position P1 in the teacher information storage unit 422 (step ST26), and proceeds to step ST27.
[0072] Next, when the leading edge of the dark drawing line IPb in the instruction image IP reaches the second set position P2, the process returns to step ST23, and the teacher information generating unit 423 acquires the position information (x2, y2) of the leading edge of the instruction image IP at the second set position P2. After that, in step ST24, the teacher information generating unit 423 sets the first transparent antenna 21 to the Nth transparent antenna 2 at the second setting position P2. N When each of the first transparent antenna 21 to the Nth transparent antenna 2 is used as a transmitting antenna, N The (N×N) pieces of reception information S(t, g, h) generated by the reception information generator 33 are obtained from the reception signals received by the .
[0073] In step ST25, the teacher information generation unit 423 generates teacher information S(t, g, h|x2, y2) at the second setting position P2 by pairing the acquired position information (x2, y2) of the second setting position P2 with the acquired (N×N) pieces of received information S(t, g, h) at the second setting position P2, and in step ST26, stores the teacher information S(t, g, h|x2, y2) at the second setting position P2 generated by the teacher information generation unit 423 in the teacher information storage unit 422, and proceeds to step ST27.
[0074] In this way, the leading edge of the dark line IPb in the instruction image IP is the Dth setting position P D From the first set position P1 to the Dth set position P D The loop from step ST23 to step ST27 is repeated for each set position P d The teacher information S(t, g, h|x d , y d ) is stored in the teacher information storage unit 422. In step ST27, the set position P d In other words, the positions are set over the entire area of the instruction image IP on the display screen of the panel 10, that is, the positions are set from the first set position P1 to the Dth set position P D When the teacher information S(t, g, h|x, y) is generated for all and stored in the teacher information storage unit 422, the operation of generating the teacher information is completed.
[0075] In steps ST21 to ST27, the teacher information generating unit 423 provides the panel 10 with instruction image information for displaying an instruction image IP in which a finger is placed on the display screen of the panel 10, and transmits the reflected wave from the finger that is placed along the instruction image IP displayed on the display screen of the panel 10 to the first transparent antenna 21 to the Nth transparent antenna 2. NThis is a step in which a received signal is received from the transparent antenna, and based on the received signal, received information S(t, g, h) corresponding to the transparent antenna that received the reflected wave is used to generate teacher information S(t, g, h|x, y) to be stored in the teacher information storage unit 422, which is linked to position information (x, y) on the display screen of the panel 10 by the instruction image IP.
[0076] Furthermore, when the reception information acquisition unit 41 and the machine learning unit 42 are realized by a computer, a program for executing the processes from step ST21 to step ST27 is stored in the ROM, and the CPU executes the program stored in the ROM. The program stored in the ROM provides the panel 10 with instruction image information for displaying an instruction image IP for indicating a finger to be present on the display screen of the panel 10, and transmits the reflected waves from the first transparent antenna 21 to the Nth transparent antenna 2 to the Nth transparent antenna 3 to the Nth transparent antenna 4 to the panel 10. N The program is a teacher information generating program for estimating the position of an object on the display screen of an operation panel device, which includes a procedure for generating teacher information S(t, g, h|x, y) linked to position information (x, y) on the display screen of the panel 10 by an instruction image IP, using received information S(t, g, h) corresponding to a transparent antenna that received a reflected wave based on a received signal from the transparent antenna.
[0077] As described above, the operation panel device according to the first embodiment has the first transparent antenna 21 to the Nth transparent antenna 22 arranged two-dimensionally on the display screen of the panel 10. N The received information generating unit 33 receives a reflected wave from a finger present on the display screen of the panel 10 and outputs the reflected wave to the first transparent antenna 21 to the Nth transparent antenna 22. N Based on the received signal from the transparent antenna, the first transparent antenna 21 to the Nth transparent antenna 2 associated with the transparent antenna that received the reflected wave are NThe object position estimation unit 40 generates received information S(t, g, h) corresponding to the received information S(t, g, h) generated by the received information generation unit 33, and performs machine learning on the received information S(t, g, h|x, y) stored in the teacher information storage unit 422 to estimate the position of the finger on the display screen of the panel 10. This improves the accuracy of estimating the position of the finger even when the finger is not in contact with the display screen of the panel 10.
[0078] In the operation panel device according to the first embodiment, the teacher information generating unit 423 provides the panel 10 with instruction image information for displaying an instruction image IP in which a finger is placed on the display screen of the panel 10, and the first transparent antenna 21 to the Nth transparent antenna 22 receive a reflected wave from a finger that is placed along the instruction image IP displayed on the display screen of the panel 10. N The teacher information S(t, g, h|x, y) stored in the teacher information storage unit 422 is generated by linking the position information (x, y) on the display screen of the panel 10 by the instruction image IP to the received information S(t, g, h) corresponding to the transparent antenna that received the reflected wave based on the received signal from the antenna, so that the generation and acquisition of the teacher information S(t, g, h|x, y) can be easily and simply achieved without taking a long time. Furthermore, even if the display screen of the panel 10 becomes cracked or dirty, the teacher information S(t, g, h|x, y) can be easily acquired again, preventing deterioration of the estimation accuracy of the finger position.
[0079] Embodiment 2 An operation panel device according to the second embodiment will be described with reference to FIG. The teacher information generating unit 423 in the operation panel device according to the first embodiment receives the reflected wave from the finger from the first transparent antenna 21 to the Nth transparent antenna 2 N Based on the received signals from all of them, teacher information S(t, g, h|x, y) is generated using the received information S(t, g, h) corresponding to the transparent antenna that received the reflected wave.
[0080] In contrast, in the operation panel device according to the second embodiment, the teacher information generating unit 423 generates the first transparent antenna 21 to the Nth transparent antenna 2 NAmong them, teacher information S(t, g, h|x, y) is generated using received information S(t, g, h) corresponding to a transparent antenna that receives a reflected wave based on received signals from multiple activated antennas present around the position on the display screen of panel 10 indicated by the instruction image IP displayed on the display screen of panel 10. The operation panel device according to the second embodiment differs from the operation panel device according to the first embodiment in this respect, and is the same in other respects. In FIG. 9, the same reference numerals as those in FIGS. 1 to 8 indicate the same or corresponding parts.
[0081] The following description will focus on the differences from the operation panel device according to the first embodiment. In the operation panel device according to the second embodiment, the position estimation operation is the same as the position estimation operation in the operation panel device according to the first embodiment, so a description thereof will be omitted. Therefore, the calibration execution operation, that is, the operation of generating teacher information S(t, g, h|x, y), which is calibration result data linked to position information (x, y) on the display screen of panel 10, will be described with reference to FIG. 8.
[0082] In the operation panel device of embodiment 2, similarly to steps ST21 to ST23 in the operation panel device of embodiment 1, the teacher information generating unit 423 outputs instruction image information for displaying an instruction image IP on the display screen of the panel 10 to the panel 10 via the control unit 50 (step ST21), and as shown in FIG. 9, a thin drawing line IPa which is a zigzag trajectory is displayed on the display screen, and the leading part moves slowly along the thin drawing line IPa, and the moved part becomes a dark drawing line IPb, thereby displaying an instruction image IP indicating an instruction for the user to slowly trace with their finger, following the movement of the leading part of the dark drawing line IPb (step ST22).
[0083] Then, when the leading edge of the dark drawn line IPb in the instruction image IP reaches the first set position P1, the teacher information generating unit 423 acquires position information (x1, y1) of the leading edge of the instruction image IP at the first set position P1 (step ST23). In response to an instruction from the teacher information generating unit 423, the first transparent antenna 21 to the Nth transparent antenna 2 N Among them, a plurality of transparent antennas around the first setting position P1, for example, four transparent antennas around the first setting position P1 in this example, are activated, that is, turned on. For example, as shown by the solid line frame in FIG. d At the setting position P d Transparent antenna surrounding 2 12 , Transparent Antenna 2 13 , Transparent Antenna 2 22 , and Transparent Antenna 2 23 The four transparent antennae are activated.
[0084] The first transparent antenna 21 to the Nth transparent antenna 2 N Among them, the transparent antennas other than the four activated transparent antennas are inactivated, that is, turned off. In FIG. 9, these are indicated by dashed frames. In this example, activating and deactivating the transparent antenna is not limited to turning the transparent antenna on and off, but also includes turning the signal receiving unit corresponding to the transparent antenna on and off while keeping the transparent antenna on.
[0085] At the first setting position P1, which corresponds to step ST24 in the operation panel device according to the first embodiment, the first transparent antenna 21 to the Nth transparent antenna 2 N When each of the activated transparent antennas among the plurality of transparent antennas is used as a transmitting antenna, the received information generating unit 33 generates, for example, (4 × 4) pieces of received information S(t, g, h) based on the received signals received by the activated transparent antennas. The teacher information generating unit 423 acquires this information via the received information acquiring unit 41.
[0086] Hereinafter, similarly to steps ST25 and ST26 in the operation panel device of embodiment 1, the teacher information generation unit 423 generates teacher information S(t, g, h|x1, y1) at the first setting position P1 by pairing the acquired position information (x1, y1) of the first setting position P1 with, for example, (4×4) pieces of received information S(t, g, h) at the acquired first setting position P1, stores the generated teacher information S(t, g, h|x1, y1) at the first setting position P1 in the teacher information storage unit 422 (step ST26), and proceeds to step ST27.
[0087] Next, when the leading edge of the dark line IPb in the instruction image IP reaches the second set position P2, the teacher information generating unit 423 acquires position information (x2, y2) of the leading edge of the instruction image IP at the second set position P2, and generates teacher information S(t, g, h|x2, y2) at the second set position P2 in the same manner as the generation of teacher information S(t, g, h|x1, y1) at the first set position P1. Generate and stores it in the teacher information storage unit 422. That is, multiple transparent antennas present around the second set position P2 are activated, and received information S(t, g, h) based on the received signals from the activated transparent antennas is paired with position information (x2, y2) of the second set position P2 to generate teacher information S(t, g, h|x2, y2) at the second set position P2.
[0088] In this way, the leading edge of the dark line IPb in the instruction image IP is the Dth setting position P D From the first set position P1 to the Dth set position P D In order of setting position, d At the setting position P d The teacher information S(t, g, h|x d , y d ) are repeatedly generated and stored in the teacher information storage unit 422. The positions set over the entire area of the instruction image IP on the display screen of the panel 10, that is, the first set position P1 to the Dth set position P DWhen the teacher information S(t, g, h|x, y) is generated for all and stored in the teacher information storage unit 422, the operation of generating the teacher information is completed.
[0089] In the operation panel device according to the second embodiment, the step of generating teacher information S(t, g, h|x, y) includes the steps of: teacher information generating unit 423 provides, to panel 10, instruction image information for displaying instruction image IP for indicating a finger to be placed on the display screen of panel 10; and transmitting reflected waves reflected from a finger present along instruction image IP displayed on the display screen of panel 10 to first transparent antenna 21 to Nth transparent antenna 22. N This is a step of generating teacher information S(t, g, h|x, y) to be stored in the teacher information storage unit 422, in which position information (x, y) on the display screen of panel 10 by the instruction image IP is linked to received information S(t, g, h) corresponding to transparent antennas that are present around the position on the display screen of panel 10 indicated by the instruction image IP displayed on the display screen of panel 10, receive received signals from multiple activated antennas, and receive reflected waves based on the received received signals.
[0090] In addition, when the received information acquisition unit 41 and the machine learning unit 42 are realized by a computer, a program for executing the process of generating the teacher information S(t, g, h|x, y) to be stored in the teacher information storage unit 422 is stored in the ROM, and the CPU executes the program stored in the ROM. The program stored in the ROM provides the panel 10 with instruction image information for displaying an instruction image IP for indicating the presence of a finger on the display screen of the panel 10; The first transparent antenna 21 to the Nth transparent antenna 22 receive the reflected wave reflected from the finger present along the instruction image IP displayed on the display screen of the panel 10. NThe program is a teacher information generating program for estimating the position of an object on the display screen of an operation panel device, which includes a procedure for generating teacher information S(t, g, h|x, y) to be stored in the teacher information storage unit 422, in which position information (x, y) on the display screen of the panel 10 based on the instruction image IP is linked to received information S(t, g, h) corresponding to transparent antennas that are present around the position on the display screen of the panel 10 indicated by the instruction image IP displayed on the display screen of the panel 10, receive received signals from multiple activated antennas, and receive the reflected waves used for the received received signals.
[0091] As described above, the operation panel device according to the second embodiment has the same effects as the operation panel device according to the first embodiment. Furthermore, in the operation panel device according to the second embodiment, the teacher information generating unit 423 generates the first transparent antenna 21 to the Nth transparent antenna 22. N The teacher information S(t, g, h|x, y) stored in the teacher information storage unit 422 is generated by linking position information (x, y) on the display screen of panel 10 by the instruction image IP to received information S(t, g, h) corresponding to transparent antennas that receive reflected waves based on received signals in which multiple activated antennas receive reflected waves reflected from the finger, and the transmitted wave is radiated from the transparent antennas around the finger, and the received strength of the reflected wave reflected by the finger is high for the transparent antennas around the finger, so that the teacher information S(t, g, h|x, y) can be generated efficiently and the memory capacity of the teacher information storage unit 422 can be used efficiently.
[0092] Embodiment 3 An operation panel device according to the third embodiment will be described with reference to FIG. The operation panel device according to the third embodiment differs from the operation panel device according to the first embodiment in that it further includes a reception sensitivity determination section 43, but is otherwise the same. The reception sensitivity determination unit 43 detects the transparent antenna 2 kThe system detects the reception sensitivity of the waves reflected from the finger by the sensor, and prompts the user to reacquire the teacher information S(t, g, h|x, y) if the reception sensitivity falls below a set value. By detecting the deterioration of the reception sensitivity, it is possible to detect that dirt has adhered to the display screen of the panel 10, or Panel 10 Transparent antenna 2 due to cracks appearing on the display screen k It is possible to know the deterioration of the reception sensitivity of the reflected wave from the finger due to the above. In FIG. 10, the same reference numerals as those in FIGS. 1 to 8 indicate the same or corresponding parts.
[0093] The following description will focus on the differences from the operation panel device according to the first embodiment. In the operation panel device of embodiment 3, the position estimation operation and the calibration execution operation, i.e., the operation of generating teacher information S(t, g, h|x, y), which is calibration result data linked to position information (x, y) on the display screen of panel 10, are the same as the position estimation operation according to the flowchart shown in FIG. 7 and the calibration execution operation according to the flowchart shown in FIG. 8 in the operation panel device of embodiment 1, and therefore will not be described again.
[0094] Therefore, the reception sensitivity determination unit 43 detects the transparent antenna 2 k This section will mainly explain how the sensor detects deterioration in the reception sensitivity of the reflected wave from the finger due to the finger movement and prompts the user to reacquire the teacher information S(t, g, h|x, y) when the reception sensitivity falls below a set value. The reception sensitivity determination unit 43 acquires the reception information S(t, g, h) acquired by the reception information acquisition unit 41, in this example, (N×N) pieces of reception information S(t, g, h) per transmission cycle c.
[0095] The reception sensitivity determination unit 43 determines whether the transparent antenna 2 included in the acquired reception information S(t, g, h) kThe strength of the received signal obtained from the is detected, and based on the strength of the detected received signal, when the strength of the received signal falls below a set value, reception sensitivity degradation information for displaying an image prompting the user to re-acquire the teacher information S(t, g, h|x, y) is provided to the panel 10 via the teacher information generation unit 423 and the control unit 50, and an image prompting the user to re-acquire the teacher information is displayed on the display screen of the panel 10. The reception sensitivity determination unit 43 may provide the reception sensitivity degradation information to the panel 10 via the control unit 50 without going through the teacher information generation unit 423 .
[0096] The reception sensitivity determination unit 43 compares the strength of the received signal with the set value, for example, as follows. The maximum value of the strength of the received signal contained in each of the (N×N) pieces of received information S(t, g, h) per transmission cycle c is detected, and the detected maximum value of the strength of the received signal is compared with a set value.
[0097] A transparent antenna 2 is positioned near the position where the finger is present on the display screen of the panel 10. k A transmission wave is emitted from the transparent antenna 2 located near the position where the finger is present on the display screen of the panel 10. k When the reflected wave is received from the finger, the transparent antenna 2 located near the position of the finger that received the reflected wave is k The strength of the received signal from is generally the maximum value among the strengths of the received signals obtained from the (N×N) pieces of received information S(t, g, h).
[0098] Therefore, when the maximum value of the strength of the received signal obtained based on the (N×N) pieces of received information S(t, g, h) becomes equal to or less than the set value, it can be inferred that dirt or cracks have formed on the display screen of the panel 10 where the finger is located, and as a result, the transparent antenna 2 located near the position on the display screen of the panel 10 where the finger is located is detected. k It can be determined that the receiving sensitivity has deteriorated.
[0099] In addition, a transparent antenna 2 is positioned near the position where the finger is placed. k and a transparent antenna 2 as a receiving antenna located near the position where the finger is present.k do not necessarily coincide. Furthermore, there is not necessarily only one piece of reception information S(t, g, h) that has the maximum strength among the reception signal obtained from the (N×N) pieces of reception information S(t, g, h).
[0100] In the above example, the receiving sensitivity is determined based on the strength of the received signal, but it is also possible to detect the spectral shape of the received signal, and generate receiving sensitivity degradation information that prompts the user to re-acquire the teacher information S(t, g, h|x, y) when the spectral shape changes beyond a set value, in other words, when the spectral shape becomes smaller than the set spectral shape. The reception sensitivity determination unit 43 is a unit that receives the reception information from the reception information acquisition unit 41 and the machine learning unit 42. Together It may be incorporated into a computer. In addition, the receiving sensitivity determination unit 43 includes a transparent antenna 2 k The receiver may have a sensor for detecting the strength of a signal received from the receiver, and may obtain information on deterioration of reception sensitivity by comparing the strength of the signal received by the sensor with a set value.
[0101] When an image prompting the user to re-acquire teacher information is displayed on the display screen of panel 10, the user causes the operation panel device to acquire teacher information again. The calibration execution operation in the operation panel device, i.e., the operation of generating teacher information S(t, g, h|x, y), which is calibration result data linked to position information (x, y) on the display screen of panel 10, is similar to the operation described in embodiment 1 and is executed according to the flowchart shown in FIG. 8.
[0102] However, the first set position P1 to the first set position P D In each case, when the teacher information S(t, g, h|x, y) generated by the teacher information generating unit 423 is stored in the teacher information storage unit 422 from the teacher information generating unit 423, the teacher information S(t, g, h|x, y) is stored in the teacher information storage unit 422 from the first setting position P1 to the Dth setting position P D Each can be rewritten using the teacher information S(t, g, h|x, y).
[0103] In this way, the transparent antenna 2 k When the receiving sensitivity of the reflected wave from the finger by the sensor deteriorates, the teacher information S(t, g, h|x, y) stored in the teacher information storage unit 422 can be rewritten, so that the position of the finger can be estimated using the optimal teacher information S(t, g, h|x, y) according to the state of the display screen of the panel 10.
[0104] As described above, the operation panel device according to the third embodiment has the same effects as the operation panel device according to the first embodiment. Furthermore, in the operation panel device according to the third embodiment, the reception sensitivity determination unit 43 determines whether the first transparent antenna 21 to the Nth transparent antenna 2 N If the receiving sensitivity of the received signal from is below a set value, receiving sensitivity degradation information is provided to panel 10, which causes an image to be displayed on the display screen of panel 10 to prompt the user to acquire teacher information S(t, g, h|x, y). This makes it possible to rewrite the teacher information S(t, g, h|x, y) stored in the teacher information storage unit 422 according to the state of the display screen of panel 10, and to estimate the position of the finger using the optimal teacher information S(t, g, h|x, y) according to the state of the display screen of panel 10.
[0105] Embodiment 4 An operation panel device according to the fourth embodiment will be described with reference to FIG. The operation panel device according to the fourth embodiment differs from the operation panel device according to the first embodiment in that it further includes an elapsed time determination unit 44, but is otherwise the same. In FIG. 11, the same reference numerals as those in FIGS. 1 to 8 designate the same or corresponding parts.
[0106] Due to the influence of the environment in which the operation panel device is used, after a long time, the first transparent antenna 21 to the Nth transparent antenna 22 that have received the reflected wave from the finger may become N The received information S(t, g, h) from the received signal from the position estimation unit 421 may be affected, and the accuracy of machine learning using the teacher information S(t, g, h|x, y) stored in the teacher information storage unit 422 by the position estimation unit 421 may decrease. Taking this into consideration, the elapsed time determination unit 44 generates all of the teacher information S(t, g, h|x, y) to be stored in the teacher information storage unit 422 by the teacher information generation unit 423, and when the elapsed time from the time when the teacher information was stored in the teacher information storage unit 422 reaches or exceeds a set time, prompts the user to obtain the teacher information S(t, g, h|x, y) on the display screen of the panel 10.
[0107] The following description will focus on the differences from the operation panel device according to the first embodiment. In the operation panel device of embodiment 4, the position estimation operation and the calibration execution operation, i.e., the operation of generating teacher information S(t, g, h|x, y), which is calibration result data linked to position information (x, y) on the display screen of panel 10, are the same as the position estimation operation according to the flowchart shown in FIG. 7 and the calibration execution operation according to the flowchart shown in FIG. 8 in the operation panel device of embodiment 1, and therefore will not be described again.
[0108] Therefore, the following will mainly explain how the elapsed time determination unit 44 prompts the re-acquisition of teacher information S(t, g, h|x, y) when the elapsed time reaches or exceeds a set time after the teacher information generation unit 423 has completed the operation of generating teacher information S(t, g, h|x, y). The elapsed time determination unit 44 receives the time when the teacher information generation unit 423 completes the operation of generating the teacher information S(t, g, h|x, y) (hereinafter referred to as the calibration completion time) from the teacher information generation unit 423 and stores the calibration completion time.
[0109] The elapsed time determination unit 44 periodically reads the current time and compares the read current time with the stored calibration completion time. If the elapsed time from the calibration completion time to the current time is equal to or longer than a set time, it provides elapsed time image information to the panel 10 via the teacher information generation unit 423 and the control unit 50, which displays an image prompting the user to re-acquire teacher information S(t, g, h|x, y), and displays an image on the display screen of the panel 10 indicating the need to re-acquire teacher information. It should be noted that elapsed time determination section 44 may provide elapsed time image information to panel 10 via control section 50 without going through teacher information generation section 423 .
[0110] In the above example, the elapsed time is obtained using the calibration completion time and the current time, but it is also possible to have a timer as the elapsed time determination unit 44, set the timer based on the calibration completion time, and output elapsed time image information when the elapsed time based on the timer reaches the set time. The elapsed time determination unit 44 is a unit that determines whether or not the received information is received by the reception information acquisition unit 41 and the machine learning unit 42. Together It may be incorporated into a computer.
[0111] When an image prompting the user to re-acquire teacher information is displayed on the display screen of panel 10, the user causes the operation panel device to acquire teacher information again. The calibration execution operation in the operation panel device, i.e., the operation of generating teacher information S(t, g, h|x, y), which is calibration result data linked to position information (x, y) on the display screen of panel 10, is similar to the operation described in embodiment 1 and is executed according to the flowchart shown in FIG. 8.
[0112] However, the first set position P1 to the Dth set position P D In each case, when the teacher information S(t, g, h|x, y) generated by the teacher information generating unit 423 is stored in the teacher information storage unit 422 from the teacher information generating unit 423, the teacher information S(t, g, h|x, y) is stored in the teacher information storage unit 422 from the first setting position P1 to the Dth setting position P D Each can be rewritten using the teacher information S(t, g, h|x, y).
[0113] Furthermore, when the teacher information generating unit 423 completes the operation of reacquiring the teacher information S(t, g, h|x, y), the elapsed time determining unit 44 receives the calibration completion time from the teacher information generating unit 423 and rewrites the calibration completion time. If the elapsed time determination unit 44 has a timer, the time on the timer is reset.
[0114] As described above, the operation panel device according to the fourth embodiment has the same effects as the operation panel device according to the first embodiment. Furthermore, in the operation panel device of embodiment 4, the teacher information generation unit 423 generates teacher information S(t, g, h|x, y) to be stored in the teacher information storage unit 422, and when the elapsed time from the time stored in the teacher information storage unit 422 becomes equal to or greater than a set time, elapsed time image information is provided to the panel 10 to display an image on the display screen of the panel 10 prompting the user to obtain the teacher information S(t, g, h|x, y).This makes it possible to rewrite the teacher information S(t, g, h|x, y) stored in the teacher information storage unit 422 every time the set time elapses, and allows the finger position to be estimated using the optimal teacher information S(t, g, h|x, y) taking into account the influence of the usage environment of the operation panel device.
[0115] Embodiment 5. An operation panel device according to the fifth embodiment will be described with reference to FIG. The operation panel device according to the fifth embodiment differs from the operation panel device according to the first embodiment in that it further includes a device temperature determination unit 45 and a device temperature measurement unit 46, but is otherwise the same. In FIG. 12, the same reference numerals as those in FIGS. 1 to 8 indicate the same or corresponding parts.
[0116] In the operation panel device, when the temperature change of the device exceeds a set value, the first transparent antenna 21 to the Nth transparent antenna 2 N When the temperature drift of the device relative to the finger reaches or exceeds a specified value, the first transparent antenna 21 to the Nth transparent antenna 2 N The received information S(t, g, h) from the received signal from the position estimation unit 421 is affected, and the accuracy of machine learning using the teacher information S(t, g, h|x, y) stored in the teacher information storage unit 422 by the position estimation unit 421 decreases.
[0117] Taking this into consideration, the teacher information generation unit 423 generates all the teacher information S(t, g, h|x, y) to be stored in the teacher information storage unit 422. When the difference between the temperature of the device, in this example the temperature of the panel 10, and the temperature of the device when stored in the teacher information storage unit 422, in this example the temperature of the panel 10, becomes equal to or exceeds a set value, the device temperature determination unit 45 prompts the acquisition of the teacher information S(t, g, h|x, y) on the display screen of the panel 10.
[0118] The following description will focus on the differences from the operation panel device according to the first embodiment. In the operation panel device of embodiment 5, the position estimation operation and the calibration execution operation, i.e., the operation of generating teacher information S(t, g, h|x, y), which is calibration result data linked to position information (x, y) on the display screen of panel 10, are the same as the position estimation operation according to the flowchart shown in FIG. 7 and the calibration execution operation according to the flowchart shown in FIG. 8 in the operation panel device of embodiment 1, and therefore will not be described.
[0119] Therefore, the following description will be focused on the point where the device temperature determination unit 45 generates the teacher information S(t, g, h|x, y) that is stored in the teacher information storage unit 422 by the teacher information generation unit 423, compares the difference between the temperature of the panel 10 at the time the teacher information is stored in the teacher information storage unit 422 and the temperature of the panel at the set time when the position estimation operation is performed, and prompts the re-acquisition of the teacher information S(t, g, h|x, y) if the difference between the two temperatures becomes equal to or greater than the set difference value.
[0120] An apparatus temperature measuring unit 46 detects the temperature of the apparatus, in this example, the temperature of the panel 10 . The teacher information generation unit 423 generates teacher information S(t, g, h|x, y) to be stored in the teacher information storage unit 422, and acquires the temperature of the panel 10 detected by the device temperature measurement unit 46 at the time of storing the teacher information in the teacher information storage unit 422, i.e., at the time of completion of the calibration execution operation, and periodically after a set time has elapsed after completion of the calibration execution operation, and provides the temperature to the device temperature determination unit 45.
[0121] Device temperature determination section 45 stores the temperature of panel 10 at the time when the calibration execution operation is completed, which is received from teacher information generation section 423, as a set value. Device temperature determination section 45 uses the temperature of panel 10 periodically acquired by teacher information generation section 423 as a comparison value, and compares the comparison value with a stored setting value. When the difference between the set value and the comparison value is greater than or equal to the difference set value, the device temperature judgment unit 45 provides temperature image information to the panel 10 via the teacher information generation unit 423 and the control unit 50, which displays an image prompting the user to re-acquire the teacher information S(t, g, h|x, y), and displays an image on the display screen of the panel 10 indicating the need to re-acquire the teacher information.
[0122] Although device temperature determination section 45 obtains the temperature of panel 10 detected by device temperature measurement section 46 via teacher information generation section 423 , it may obtain the temperature directly from device temperature measurement section 46 . Furthermore, device temperature determination section 45 may provide temperature image information to panel 10 via control section 50 without going through teacher information generation section 423 .
[0123] When an image prompting the user to re-acquire teacher information is displayed on the display screen of panel 10, the user causes the operation panel device to acquire teacher information again. The calibration execution operation in the operation panel device, i.e., the operation of generating teacher information S(t, g, h|x, y), which is calibration result data linked to position information (x, y) on the display screen of panel 10, is similar to the operation described in embodiment 1 and is executed according to the flowchart shown in FIG. 8.
[0124] However, the first set position P1 to the Dth set position P D In each case, when the teacher information S(t, g, h|x, y) generated by the teacher information generating unit 423 is stored in the teacher information storage unit 422 from the teacher information generating unit 423, the teacher information S(t, g, h|x, y) is stored in the teacher information storage unit 422 from the first setting position P1 to the Dth setting position P D Each can be rewritten using the teacher information S(t, g, h|x, y).
[0125] In addition, when the teacher information generation unit 423 completes the operation of re-acquiring the teacher information S(t, g, h|x, y), the device temperature determination unit 45 rewrites the stored setting value for the temperature of the panel 10 to the temperature of the panel 10 acquired by the teacher information generation unit 423 from the device temperature measurement unit 46 at the time of completion of the re-acquisition operation, and stores it as a new setting value.
[0126] As described above, the operation panel device according to the fifth embodiment has the same effects as the operation panel device according to the first embodiment. Furthermore, in the operation panel device according to the fifth embodiment, device temperature determination section 45 determines the temperature of panel 10 based on a set value of the temperature of panel 10. Compare When the difference between the two temperatures becomes equal to or greater than the difference setting value, an image prompting the user to obtain teaching information S(t, g, h|x, y) is displayed on the display screen of the panel 10. temperature Since image information is provided to the panel 10, the first transparent antenna 21 to the Nth transparent antenna 2 N The device temperature drift for the operation panel device can be suppressed within a specified value, the device is less susceptible to the temperature of the environment in which the device is used, and the finger position can be estimated using the teacher information S(t, g, h|x, y).
[0127] In addition, any one of the receiving sensitivity determination unit 43 shown in embodiment 3, the elapsed time determination unit 44 shown in embodiment 4, or the device temperature determination unit 45 shown in embodiment 5 may be applied to the operation panel device of embodiment 2. Furthermore, the embodiments may be freely combined, or any of the components in each embodiment may be modified, or any of the components in each embodiment may be omitted. [Industrial Applicability]
[0128] The operation panel device according to the present disclosure is suitable for use as a non-contact operation panel device as a user interface in various electronic devices having a touch display function, such as mobile terminals and portable devices. The present invention is also suitable for use in an operation panel device that is used in an environment where the display screen of the panel is likely to become dirty, such as in a factory, where the device is used by users with dirty fingers. [Explanation of symbols]
[0129] 10 Display Panel, 20 Transparent Antenna Array, 2 11 ~2 mn , 21~2 N The first transparent antenna to the Nth transparent antenna, 30 a transmission / reception circuit unit, 31 an input / output switching unit, 311 to 31 N The first switching unit to the Nth switching unit, 32 signal transmission unit, 321 signal generator, 322 output destination selection unit, 33 reception information generation unit, 331 to 33 N First signal receiving unit to Nth signal receiving unit, 40 object position estimation unit, 41 received information acquisition unit, 42 machine learning unit, 421 position estimation unit, 422 teacher information storage unit, 423 teacher information generation unit, 43 receiving sensitivity determination unit, 44 elapsed time determination unit, 45 device temperature determination unit, 50 control unit.
Claims
1. A display panel having a display screen; a plurality of transparent antennas arranged two-dimensionally on a display screen of the display panel; a reception information generating unit that generates reception information corresponding to the plurality of transparent antennas that receive a reflected wave, the reception information being associated with the transparent antenna that receives the reflected wave, based on reception signals from the plurality of transparent antennas that receive a reflected wave from an object present on a display screen of the display panel; a storage unit for storing teacher information corresponding to the plurality of transparent antennas linked to position information on a display screen of the display panel; a position estimation unit that performs machine learning on the received information generated by the received information generation unit using teacher information stored in the storage unit, and estimates a position of the object on a display screen of the display panel; a teacher information generating unit that generates teacher information to be stored in the storage unit, the teacher information generating unit generating teacher information in which position information on the display screen of the display panel according to the instruction image is linked to reception information corresponding to the transparent antennas that received the reflected waves based on reception signals from the transparent antennas that received reflected waves from the object that exists along the instruction image displayed on the display screen of the display panel; and An operation panel device comprising:
2. A display panel having a display screen; a plurality of transparent antennas arranged two-dimensionally on a display screen of the display panel; a reception information generating unit that generates reception information corresponding to the plurality of transparent antennas that receive a reflected wave, the reception information being associated with the transparent antenna that receives the reflected wave, based on reception signals from the plurality of transparent antennas that receive a reflected wave from an object present on a display screen of the display panel; a storage unit for storing teacher information corresponding to the plurality of transparent antennas linked to position information on a display screen of the display panel; a position estimation unit that performs machine learning on the received information generated by the received information generation unit using teacher information stored in the storage unit, and estimates a position of the object on a display screen of the display panel; a teacher information generating unit that provides the display panel with instruction image information for displaying an instruction image indicating the presence of the object on a display screen of the display panel, activates a plurality of transparent antennas among the plurality of transparent antennas that are present around a position on the display screen of the display panel that is indicated by the instruction image displayed on the display screen of the display panel, and generates teacher information to be stored in the storage unit, in which position information on the display screen of the display panel that is indicated by the instruction image is linked to reception information corresponding to the transparent antenna that received the reflected wave using reception signals from the activated plurality of transparent antennas that have received a reflected wave from the object that is present along the instruction image displayed on the display screen of the display panel; An operation panel device comprising:
3. A display panel having a display screen; a plurality of transparent antennas arranged two-dimensionally on a display screen of the display panel; a reception information generating unit that generates reception information corresponding to the plurality of transparent antennas that receive a reflected wave, the reception information being associated with the transparent antenna that receives the reflected wave, based on reception signals from the plurality of transparent antennas that receive a reflected wave from an object present on a display screen of the display panel; a storage unit for storing teacher information corresponding to the plurality of transparent antennas linked to position information on a display screen of the display panel; a position estimation unit that performs machine learning on the received information generated by the received information generation unit using teacher information stored in the storage unit, and estimates a position of the object on a display screen of the display panel; a teacher information generating unit that generates teacher information to be stored in the storage unit, the teacher information generating unit providing the display panel with instruction image information for displaying an instruction image indicating a trajectory along which the object will be moved on a display screen of the display panel, and linking position information on the display screen of the display panel according to the instruction image to reception information corresponding to the transparent antennas that received the reflected waves based on reception signals from the transparent antennas that received the reflected waves from the object that is moved along the trajectory of the instruction image displayed on the display screen of the display panel; An operation panel device comprising:
4. The operation panel device according to claim 1 , wherein the machine learning performed by the position estimation unit uses a machine learning method selected from deep learning and Gaussian process regression.
5. The operation panel device according to any one of claims 1 to 3, further comprising a reception sensitivity determination unit that provides the display panel with reception sensitivity degradation information that displays an image on the display screen prompting the user to obtain teacher information when the reception sensitivity of the received signals from the plurality of transparent antennas is below a set value.
6. The operation panel device according to any one of claims 1 to 3, further comprising an elapsed time determination unit that provides elapsed time image information to the display panel, the elapsed time image information causing an image to be displayed on the display screen prompting the user to obtain the teacher information when the teacher information generation unit generates teacher information to be stored in the storage unit and the elapsed time from the time stored in the storage unit becomes equal to or exceeds a set time.
7. An operation panel device as described in any one of claims 1 to 3, further comprising a device temperature determination unit that provides the display panel with temperature image information causing an image to be displayed on the display screen prompting the user to obtain teacher information when the difference between the temperature of the operation panel device and the temperature of the operation panel device when the teacher information generation unit generates the teacher information to be stored in the storage unit becomes equal to or greater than a set value.
8. A step in which a reception information generating unit receives reception signals from a plurality of transparent antennas arranged two-dimensionally on the display screen of a display panel that receive reflected waves reflected from an object present on the display screen of the display panel, and generates reception information corresponding to the plurality of transparent antennas linked to the transparent antenna that received the reflected waves based on the received reception signals; a position estimation unit performing machine learning on the received information generated by the received information generation unit using teacher information stored in a storage unit, and estimating a position of the object on a display screen of the display panel; a teacher information generating unit providing the display panel with instruction image information for displaying an instruction image indicating the presence of the object on a display screen of the display panel, receiving received signals from the plurality of transparent antennas that have received reflected waves from the object that exists along the instruction image displayed on the display screen of the display panel, and generating teacher information to be stored in the storage unit in which position information on the display screen of the display panel by the instruction image is linked to reception information corresponding to the transparent antennas that have received the reflected waves based on the received reception signals; An object position estimation method for estimating a position of an object on a display screen of an operation panel device comprising:
9. A step of acquiring reception information corresponding to a plurality of transparent antennas linked to the transparent antenna that received the reflected wave, based on reception signals from the plurality of transparent antennas arranged two-dimensionally on the display screen of a display panel that receives a reflected wave from an object present on the display screen of the display panel; a step of performing machine learning using teacher information on the acquired received information to estimate a position of the object on a display screen of the display panel; providing, to the display panel, instruction image information for displaying an instruction image indicating the presence of the object on a display screen of the display panel, and generating teacher information in which position information on the display screen of the display panel according to the instruction image is linked to reception information corresponding to the transparent antennas that received the reflected waves, based on reception signals from the plurality of transparent antennas that received reflected waves from the object that exists along the instruction image displayed on the display screen of the display panel; A program for estimating an object position on a display screen of an operation panel device, the program causing a computer to execute the above steps.
10. A step of acquiring reception information corresponding to a plurality of transparent antennas linked to the transparent antenna that received the reflected wave, based on reception signals from the plurality of transparent antennas arranged two-dimensionally on the display screen of a display panel that receives a reflected wave from an object present on the display screen of the display panel; a step of performing machine learning using teacher information on the acquired received information to estimate a position of the object on a display screen of the display panel; providing, to the display panel, instruction image information for displaying an instruction image indicating the presence of the object on a display screen of the display panel, and generating teacher information in which position information on the display screen of the display panel according to the instruction image is linked to reception information corresponding to the transparent antennas that received the reflected waves, based on reception signals from the plurality of transparent antennas that received reflected waves from the object that exists along the instruction image displayed on the display screen of the display panel; A recording medium storing a program for estimating the position of an object on a display screen of an operation panel device, the program causing a computer to execute the above.
Citation Information
Patent Citations
Hand-holdable electronic devices and method of determining an electromagnetic environment near to such a device
EP3407175A1
SENSING SCREEN, CONTROL CIRCUIT AND CONTROL METHOD THEREOF, AND SENSING SCREEN DEVICE
JP2017527146A
Position detection device
JP2020144647A
Smartphone with radar system, system and method
JP2021504771A
Devices, systems, and methods for detecting gestures using multiple antennas and / or reflections of signals transmitted by the detecting device
US20170131395A1
