Interaction control device and interaction system

Abstract

An interactive control device and an interactive system, the interactive control device comprising: an antenna array for transmitting radar signals and receiving reflected echo signals; a data processing module for determining human body actions and corresponding control instructions according to the radar signals and the echo signals, and outputting the control instructions to a terminal device; and a display module for displaying the determined human body actions and / or the control instructions.

Description

Technical Field

[0001] This disclosure relates to the field of human-computer interaction technology, and in particular to an interactive control device and an interactive system. Background Technology

[0002] With the rapid development of 5G mobile communication and IoT technologies, intelligent interactive control is also entering a period of rapid development. Through the internet, various devices in office and home environments can be connected and controlled, providing information exchange. Currently, human-computer interaction control is almost entirely based on contact buttons or touch operations. There are no effective solutions for situations where contact control is inconvenient, or where interactive operations are unfriendly and lack flexibility. Summary of the Invention

[0003] This disclosure aims to solve at least one of the technical problems existing in the prior art, and proposes an interactive control device and interactive system.

[0004] To achieve the above objectives, in a first aspect, embodiments of this disclosure provide an interactive control device, comprising:

[0005] Antenna arrays are used to transmit radar signals and receive reflected echo signals.

[0006] The data processing module is used to determine human body movements and corresponding control commands based on the radar signals and the echo signals, and to output the control commands to the terminal device.

[0007] The display module is used to display the determined human body movements and / or the control commands.

[0008] In some embodiments, the data processing module is specifically used to mix and analyze the radar signal and the echo signal to obtain reflecting object information, and to identify the human body movement based on the reflecting object information and determine the corresponding control command; wherein the reflecting object information includes at least one of distance information, speed information and angle of arrival information.

[0009] In some embodiments, the radar signal is a frequency-modulated continuous wave signal.

[0010] In some embodiments, the data processing module includes: a first processing core and a second processing core;

[0011] The first processing core is used to perform analysis and calculation on the signal obtained after mixing the radar signal and the echo signal to generate the information of the reflecting object. The analysis and calculation includes at least one of one-dimensional Fourier transform, two-dimensional Fourier transform, and angle of arrival calculation.

[0012] The second processing core is used to perform chirp control on the radar signal, pre-train and generate a human motion recognition network, and identify the human motion through the human motion recognition network based on the information of the reflecting object, and determine the corresponding control command.

[0013] In some embodiments, the data processing module includes: a low-noise amplifier, a mixer, an intermediate frequency amplifier, an analog-to-digital converter, a digital front-end component, a buffer, a power amplifier, a power divider, and a waveform generator;

[0014] The low-noise amplifier, the mixer, the intermediate frequency amplifier, the analog-to-digital converter, the digital front-end component, and the buffer are connected in sequence.

[0015] The waveform generator, the power divider, and the power amplifier are connected in sequence;

[0016] The input of the mixer is also connected to the power divider, the input of the low-noise amplifier is connected to the antenna array, and the output of the power amplifier is connected to the antenna array.

[0017] In some embodiments, the display module includes a display, and the antenna array is integrated in the display. The antenna array includes a reference electrode layer, a dielectric layer, and a radiating electrode layer stacked sequentially.

[0018] In some embodiments, the display has a display area and a peripheral area surrounding the display area; the antenna array is integrated in the peripheral area, and both the reference electrode layer and the radiating electrode layer include planar electrodes.

[0019] In some embodiments, the display has a display area and a peripheral area surrounding the display area; the antenna array is integrated in the display area, and both the reference electrode layer and the radiating electrode layer include a metal mesh.

[0020] In some embodiments, the display is a liquid crystal display, and the display includes a first display panel, a first polarizer located on the display surface side of the first display panel, and a second polarizer located on the side of the first display panel opposite to the first polarizer.

[0021] In some embodiments, the antenna array is attached to the side of the first polarizer that is away from the display surface of the first display panel.

[0022] In some embodiments, the first display panel includes a color filter layer, a liquid crystal layer, and a back panel that are stacked sequentially.

[0023] The antenna array is integrated between the first polarizer and the color filter layer.

[0024] In some embodiments, the display further includes a touch layer; the touch layer is disposed between the color filter layer and the antenna array, or the touch layer is disposed between the color filter layer and the liquid crystal layer.

[0025] In some embodiments, the display is an OLED display, and the display includes a second display panel and a third polarizer located on the display surface side of the second display panel.

[0026] In some embodiments, the antenna array is attached to the side of the third polarizer that is away from the display surface of the second display panel.

[0027] In some embodiments, the second display panel includes a thin-film encapsulation layer and an OLED layer stacked together;

[0028] The antenna array is integrated between the third polarizer and the thin-film encapsulation layer.

[0029] In some embodiments, the display further includes a touch layer; the touch layer is disposed between the thin-film encapsulation layer and the antenna array, or the touch layer is disposed between the third polarizer and the thin-film encapsulation layer, and the antenna array is integrated into the touch layer.

[0030] In some embodiments, the interactive control device further includes: a printed circuit board;

[0031] The data processing module is mounted on the printed circuit board, and the antenna array transmits signals to the data processing module via a flexible cable.

[0032] In some embodiments, the interactive control device further includes: a printed circuit board;

[0033] The data processing module is mounted on the printed circuit board, and the antenna array is encapsulated inside the data processing module.

[0034] In some embodiments, the display module includes: a routing gateway unit;

[0035] The routing gateway unit is used to receive media data;

[0036] The display module is also used to display the media data.

[0037] Secondly, embodiments of this disclosure provide an interactive system, which includes: a terminal device and an interactive control device;

[0038] The interactive control device described in the above embodiments is an interactive control device.

[0039] The terminal device is used to operate according to the control commands output by the interactive control device. Attached Figure Description

[0040] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0041] Figure 1 This is a schematic diagram of the structure of an interactive control device provided in an embodiment of the present disclosure;

[0042] Figure 2 This is a schematic diagram of the structure of a data processing module provided in an embodiment of the present disclosure;

[0043] Figure 3 An equivalent circuit diagram of another data processing module provided in an embodiment of this disclosure;

[0044] Figure 4 This is a schematic diagram of the structure of another data processing module provided in an embodiment of the present disclosure;

[0045] Figure 5 A planar schematic diagram of an antenna element provided in an embodiment of this disclosure;

[0046] Figure 6 for Figure 5 A schematic diagram of the cross-sectional structure of the antenna element shown;

[0047] Figure 7 A planar schematic diagram of another antenna element provided in an embodiment of this disclosure;

[0048] Figure 8 for Figure 7 A schematic diagram of the cross-sectional structure of the antenna element shown;

[0049] Figure 9 This is a schematic diagram of the structure of a display module provided in an embodiment of the present disclosure;

[0050] Figure 10 This is a schematic diagram of another display module provided in an embodiment of the present disclosure;

[0051] Figure 11 A plan view of a second display panel provided in an embodiment of this disclosure;

[0052] Figure 12 for Figure 11 The second display panel shown is a cross-sectional view in the AA direction. Detailed Implementation

[0053] To enable those skilled in the art to better understand the technical solutions of this disclosure, the interactive control device and interactive system provided in this disclosure will be described in detail below with reference to the accompanying drawings.

[0054] Exemplary embodiments will be described more fully below with reference to the accompanying drawings; however, these exemplary embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will enable those skilled in the art to fully understand the scope of this disclosure.

[0055] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the said feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded.

[0056] It will be understood that while the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited to these terms. These terms are used only to distinguish one element from another. Therefore, without departing from the teachings of this disclosure, the first element, first component, or first module discussed below may be referred to as a second element, second component, or second module.

[0057] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0058] Figure 1 This is a schematic diagram of the structure of an interactive control device provided in an embodiment of this disclosure. Figure 1 As shown, the interactive control device includes an antenna array, a data processing module, and a display module.

[0059] Specifically, the antenna array is used to transmit radar signals and receive reflected echo signals; the data processing module is used to determine human body movements and corresponding control commands based on radar signals and echo signals, and output the control commands to the terminal device.

[0060] The display module is used to display the determined human body movements and / or control commands.

[0061] The antenna array includes multiple antenna elements, which are divided into multiple groups. Each group of antenna elements is configured to transmit radar signals and receive echo signals.

[0062] In some embodiments, the interactive control device is configured one-to-one with the terminal device, or one-to-many, or many-to-one.

[0063] In some embodiments, the data processing module is specifically used to mix and analyze the radar signal and the echo signal to obtain information about the reflecting object, and to identify human movements based on the reflecting object information and determine the corresponding control commands. The reflecting object information includes at least one of distance information, speed information, and angle of arrival information. In some embodiments, the radar signal is a frequency-modulated continuous wave (FMCW) signal.

[0064] Specifically, millimeter-wave radars are broadly classified into pulse radars and continuous-wave radars based on the type of transmitted signal. Pulse radars transmit periodic high-frequency pulses, while continuous-wave radars transmit continuous-wave signals. Continuous-wave signals can include single-frequency continuous-wave (CW) signals or frequency-modulated (FM) continuous-wave signals. For FM continuous-wave signals, the frequency modulation methods include triangular wave, sawtooth wave, coded modulation, or noise frequency modulation, etc. In the aforementioned embodiment, when the radar signal is an FM continuous-wave signal, the antenna array transmits a frequency-changing FM continuous-wave signal within a frequency sweep period. The echo signal reflected by the object has a certain frequency difference from the transmitted radar signal. Subsequently, by measuring the frequency difference, information such as the distance between the object and the antenna array can be obtained.

[0065] In some embodiments, a correspondence between human body movements and control commands is pre-acquired or configured. This may involve establishing a one-to-one correspondence between human body movements and control commands, establishing a correspondence between human body movements and control commands of a single terminal device, or establishing a correspondence between a single human body movement and control commands of different terminal devices.

[0066] In some embodiments, the interactive control device further includes a printed circuit board (PCB). Accordingly, in some embodiments, the data processing module is disposed on the PCB, and the antenna array transmits signals to the data processing module via a flexible cable. To achieve lower signal loss, anisotropic conductive film (ACF) with relatively large gold sphere diameters (e.g., greater than 10 μm) can be used for bonding. Alternatively, in some embodiments, the data processing module is disposed on the PCB, and the antenna array is encapsulated inside the data processing module. The PCB can be a high-frequency circuit board, and the antenna array can be encapsulated inside the data processing module based on antenna-in-package (AiP) technology. In some embodiments, the interactive control device is installed inside the terminal device. In this case, an opening needs to be made at the corresponding installation location on the terminal device for the sensing area corresponding to the antenna array to reserve a transmission path, so as to avoid the metal casing of the terminal device obstructing the transmission of millimeter waves.

[0067] In some embodiments, the data processing module and the antenna array form a millimeter-wave radar subsystem, and correspondingly, the display module corresponds to the display subsystem.

[0068] In some embodiments, the display module includes a routing gateway unit; the routing gateway unit is used to receive media data; correspondingly, the display module is also used to display media data received by the routing gateway unit.

[0069] In some embodiments, the display module further includes a display, a display driver unit, a chip timing control unit, a signal adapter unit, etc. The media data received by the routing gateway unit is decoded by the signal adapter unit and then driven and controlled by the timing control unit according to a certain timing logic to form a display effect on the display.

[0070] In some embodiments, the interactive control device and the terminal device share the display module.

[0071] This disclosure provides an interactive control device that forms a millimeter-wave radar based on an antenna array and a data processing module. The millimeter-wave radar enables non-contact control. By capturing and recognizing human movements, the device determines corresponding control commands and sends them to a terminal device, causing the terminal device to execute the corresponding command response and thus achieve interactive operation.

[0072] Figure 2 This is a schematic diagram of the structure of a data processing module provided in an embodiment of this disclosure. Figure 2 As shown, this data processing module is Figure 1One specific alternative implementation of the data processing module shown is as follows: Specifically, the data processing module includes a first processing core and a second processing core.

[0073] The first processing core is used to analyze and calculate the signal obtained after mixing the radar signal and the echo signal to generate information about the reflecting object. The analysis and calculation include at least one of one-dimensional fast fourier transform (1D FFT), two-dimensional fast fourier transform (2D FFT), and angle of arrival (AOA) calculation. The distance information and velocity information can be obtained through one-dimensional and two-dimensional fourier transforms, and the angle of arrival information can be obtained through angle of arrival calculation.

[0074] In some embodiments, the first processing core is further configured to determine the valid echo signal based on the Peak Search algorithm and the Constant False-Alarm Rate (CFAR) algorithm before performing the angle of arrival calculation.

[0075] The second processing core is used to perform chirp control on the radar signal, pre-train and generate a human motion recognition network, and identify human motions through the human motion recognition network based on the information of reflecting objects, and determine the corresponding control commands. Here, chirp refers to the characteristic of a signal's instantaneous frequency changing over time. In some embodiments, the radar signal is a frequency-modulated continuous wave signal, and correspondingly, the second processing core is used to configure the chirp parameters of the frequency-modulated continuous wave signal. In some embodiments, the human motion recognition network can be independently configured as a gesture recognition network for accurate recognition of gestures. In some embodiments, the human motion recognition network is a convolutional neural network, which can be based on Torch, PyTorch, VGG, etc. Convolutional neural networks have fewer parameters, faster discrimination speed, and higher discrimination accuracy, making them particularly suitable for image recognition. It should be noted that the above recognition network can use a convolutional neural network model or other neural network models, all of which are applicable to the technical solutions of this application, and will not be elaborated further here.

[0076] In some embodiments, the first processing core may be a DSP processing core, and the second processing core may be an ARM processing core.

[0077] Figure 3 An equivalent circuit diagram of another data processing module provided in an embodiment of this disclosure. For example... Figure 3 As shown, this data processing module is Figure 1The data processing module shown is a specific alternative implementation. Specifically, the data processing module includes: a low-noise amplifier 201, a mixer 202, an intermediate frequency amplifier 203, an analog-to-digital converter 204, a digital front-end component 205 (sampling filter), a buffer 206, a power amplifier 301, a power divider 302, and a waveform generator 303. The arrows in the figure indicate the signal transmission direction.

[0078] The low-noise amplifier 201, mixer 202, intermediate frequency amplifier 203, analog-to-digital converter 204, digital front-end component 205, and buffer 206 are connected in sequence; the waveform generator 303, power divider 302, and power amplifier 301 are connected in sequence; the input terminal of mixer 202 is connected to both the low-noise amplifier 201 and the power divider 302; the input terminal of low-noise amplifier 201 is connected to the antenna array, and the output terminal of power amplifier 301 is connected to the antenna array; the data processing module may include multiple receivers... The transmitting link, corresponding to the receiving side, may include multiple sets of low-noise amplifiers 201, mixers 202, intermediate frequency amplifiers 203, and analog-to-digital converters 204 (two sets are shown in the figure as an example), and corresponding to the transmitting side, may include multiple power amplifiers 301 (two are shown in the figure as an example); in some embodiments, each power amplifier 301 is also connected to a phase shifter between itself and the power divider 302; in some embodiments, each mixer 202 is also connected to a filter between itself and its corresponding intermediate frequency amplifier 203.

[0079] The waveform generator 303 generates radar signals, a portion of which is sent to the mixer 202 via the power divider 302, and another portion is sent to the antenna array via the power amplifier 301 via the power divider 302, and then transmitted outward by the corresponding antenna elements in the antenna array. The corresponding antenna elements in the antenna array receive the echo signals reflected after the radar signals encounter objects. The received echo signals are amplified by the low-noise amplifier 201, and then mixed with a portion of the output of the power divider 302 via the mixer 202 to obtain an intermediate frequency signal. This signal is then converted into corresponding data via the intermediate frequency amplifier 203, the analog-to-digital converter 204, and the digital front-end component 205, and stored in the buffer 206.

[0080] Figure 4 This is a schematic diagram of the structure of another data processing module provided in an embodiment of this disclosure. For example... Figure 4 As shown, this data processing module is based on Figure 2 and Figure 3One specific alternative implementation of the data processing module shown includes: a low-noise amplifier, a mixer, an intermediate frequency amplifier, an analog-to-digital converter, a digital front-end component, a buffer, a power amplifier, a power divider, a waveform generator, a first processing core, and a second processing core.

[0081] Based on the functions of each component, this data processing module can be divided into multiple units, including: an RF / Analog circuit (RF / Analog) unit, a transmit / receive (TR) unit, a signal processing (DSP) unit, and a control (Master) unit; such as Figure 4 As shown, the RF / analog circuit unit includes a low-noise amplifier, mixer, intermediate frequency amplifier, analog-to-digital converter, power amplifier, and power divider. The transmit / receive unit includes digital front-end components and a waveform generator. The signal processing unit includes a buffer and a first processing core. The main control unit includes a second processing core. The main control unit and the components in the signal processing unit can communicate and control processes based on a bus matrix. The signal transmission direction within the RF / analog circuit unit can be seen in [reference needed]. Figure 3 .

[0082] In some embodiments, the RF / analog circuit unit further includes a general-purpose analog-to-digital converter (GPADC), an oscillator (OSC), a temperature controller, etc.; the signal processing unit further includes a cyclic redundancy check (CRC) component, a direct memory access (DMA) component, a low voltage differential signaling (LVDS) interface, a hardware-in-the-loop (HIL) component, a radar data memory, and a hardware accelerator connected to the buffer; the main control unit further includes a direct memory access component, a serial peripheral interface (SPI), a quad serial peripheral interface (QSPI), a bus interface, and a debug serial port; a mailbox module based on a mailbox synchronous communication mechanism is also provided between the main control unit and the signal processing unit.

[0083] In some embodiments, the data processing module may employ an IWR6843 chip or a VYYR7201-A0 chip, etc.

[0084] In some embodiments, the data processing module may include only a low-noise amplifier, a mixer, an intermediate frequency amplifier, a power amplifier, and a power divider, such as using a BGT60TR13 chip, while other components such as analog-to-digital converters, digital front-end components, buffers, and waveform generators need to be provided separately.

[0085] Therefore, based on the data processing modules in the above embodiments, it is possible to process corresponding signals and data based on multiple processing cores. The first processing core performs a series of analysis and calculations on the intermediate frequency signal, and the second processing core realizes human motion recognition through a trainable human motion recognition network, thereby improving the accuracy of human motion recognition.

[0086] In some embodiments, the display module includes a display, and an antenna array is integrated in the display. The antenna array includes a reference electrode layer, a dielectric layer, and a radiating electrode layer stacked sequentially.

[0087] In some embodiments, the reference electrode layer, also known as the ground layer, is connected to the ground signal (which can be a low-level DC signal). It can discharge static electricity and lightning signals generated during use, preventing the antenna from being damaged by breakdown and affecting its performance. The radiating electrode layer, also known as the radiating layer, can convert the electrical signal input through the transmission line into an electromagnetic wave signal and radiate the electromagnetic wave signal outward. Alternatively, it can convert external electromagnetic wave signals into electrical signals and output them to the terminal device through the transmission line to achieve wireless signal transmission. The dielectric layer can be a dielectric substrate, located between the reference electrode layer and the radiating electrode layer. It can be a low-loss dielectric material and serves to support the reference electrode layer and the radiating electrode layer. In some embodiments, the antenna array is integrated into the display in ways such as integration inside the screen or integration outside the screen. Integration inside the screen can be further divided into integration on the screen or integration under the screen.

[0088] In some embodiments, the materials of the reference electrode layer and the radiation electrode layer include aluminum, copper, or molybdenum.

[0089] In some embodiments, the display has a display area and a peripheral area surrounding the display area; wherein the antenna array can be disposed at any position in the display area and the peripheral area. In some embodiments, in order to reduce signal transmission loss, the antenna array is disposed near the edge of the display to shorten the length of the signal lead-out line.

[0090] Figure 5 This is a planar schematic diagram of an antenna element provided in an embodiment of the present disclosure. Figure 6 for Figure 5 The diagram shows a cross-sectional view of the antenna element. The antenna array comprises multiple antenna elements; this corresponds to a specific optional implementation where the antenna array is integrated outside the screen. Specifically, the antenna array is integrated in the peripheral area, such as... Figure 5 and Figure 6 As shown, the antenna unit includes a reference electrode layer 401, a dielectric layer 402, and a radiating electrode layer 403. Both the reference electrode layer 401 and the radiating electrode layer 403 include planar electrodes. The antenna unit also includes a feed line 404, which is connected to the radiating electrode layer 403. The feed line 404 can transmit electrical signals generated by the data processing module to the radiating electrode layer 403, or transmit electromagnetic signals received by the radiating electrode layer 403 to the data processing module, thereby realizing signal transmission. In some embodiments, the feed line 404 can be disposed on the same layer as the radiating electrode layer 403.

[0091] In some embodiments, the antenna element may, exemplarily, have the following side length settings: ab is 0.15 mm, ac is 0.01 mm, cd is 0.3 mm, ef is 1.9 mm, and eg is 2.5 mm. It should be noted that the above dimensional illustrations are merely a specific optional implementation provided by this disclosure, and the lengths of the sides in the drawings do not strictly correspond to the above illustrations. In practice, a suitable side length setting scheme can be selected according to actual needs.

[0092] Figure 7 This is a planar schematic diagram of another antenna element provided in an embodiment of this disclosure. Figure 8 for Figure 7 The diagram shows a cross-sectional view of the antenna unit. The antenna array includes multiple antenna units; this corresponds to a specific optional implementation where the antenna array is integrated within the screen. Specifically, the antenna array is integrated into the display area, such as... Figure 7 and Figure 8 As shown, the antenna unit can be made of cyclic olefin polymer film (COP) substrate, which includes a reference electrode layer 401, a dielectric layer 402 and a radiating electrode layer 403. Both the reference electrode layer 401 and the radiating electrode layer 403 include a metal mesh with a linewidth of 10 micrometers and a side length of 150 micrometers. The mesh can be made of copper. The antenna unit also includes a feed line 404.

[0093] In some embodiments, the antenna element may, exemplarily, have the following side length settings: 'hi' has a side length of 0.25 mm, 'hj' has a side length of 0.45 mm, 'jk' has a side length of 0.3 mm, 'ml' has a side length of 1.15 mm, and 'mn' has a side length of 1.7 mm. It should be noted that the above-described dimensional illustrations are merely a specific optional implementation provided by this disclosure, and the lengths of the sides in the accompanying drawings do not strictly correspond to the above illustrations. In practice, a suitable side length setting scheme can be selected according to actual needs.

[0094] Figure 9 This is a schematic diagram of a display module provided in an embodiment of the present disclosure. Wherein, based on... Figure 7 and Figure 8The antenna array shown corresponds to a specific optional implementation scheme integrated within the screen. Specifically, the display is a liquid crystal display, such as... Figure 9 As shown, the display includes a first display panel, a first polarizer 6 located on the display surface side of the first display panel, and a second polarizer 2 located on the side of the first display panel opposite to the first polarizer 6. The first polarizer 6 is provided to reduce the reflection of ambient light. In some embodiments, as shown, the display also includes a backlight located on the side of the second polarizer opposite to the first display panel. The first display panel includes a color filter layer 5, a liquid crystal layer 4, and a backplate 3 stacked sequentially.

[0095] See Figure 9 As shown in (a), in a specific alternative embodiment integrated on the screen, the antenna array 7 is attached to the side of the first polarizer 6 away from the display surface of the first display panel; as shown in (b), in a specific alternative embodiment integrated under the screen, the antenna array 7 is integrated between the first polarizer 6 and the color filter layer 5, wherein integrating the antenna array 7 on the side of the first polarizer 6 close to the display surface of the first display panel can reduce external interference based on the characteristic of the first polarizer 6 to reduce ambient light reflection.

[0096] In some embodiments, the display further includes a touch layer 8; corresponding to a specific optional implementation integrated under the screen, as shown in (c), the touch layer 8 is disposed between the color filter layer 5 and the antenna array 7, or, as shown in (d), the touch layer 8 is disposed between the color filter layer 5 and the liquid crystal layer 4. Thus, the interactive control device provided by the embodiments of this disclosure can not only achieve contactless interactive control based on the antenna array integrated in the screen, but also is compatible with the method of controlling the terminal device via touch.

[0097] Figure 10 This is a schematic diagram of another display module provided in an embodiment of the present disclosure. Wherein, based on... Figure 7 and Figure 8 The antenna array shown corresponds to a specific alternative implementation scheme integrated within the screen. Specifically, the display is an OLED (Organic Light Emitting Diode) display, such as... Figure 10 As shown, the display includes a second display panel and a third polarizer 40 located on the display surface side of the second display panel; in some embodiments, as shown, the second display panel includes a thin film encapsulation layer 30 and an OLED layer 20 stacked together.

[0098] See Figure 10As shown in (a), in a specific alternative embodiment integrated on the screen, the antenna array 50 is attached to the side of the third polarizer 40 away from the display surface of the second display panel; as shown in (b), in a specific alternative embodiment integrated under the screen, the antenna array 50 is integrated between the third polarizer 40 and the thin film encapsulation layer 30.

[0099] In some embodiments, the display further includes a touch layer 60; as shown in (c), the touch layer 60 is disposed between the thin film encapsulation layer 30 and the antenna array 50, or, as shown in (d), the touch layer is disposed between the third polarizer 40 and the thin film encapsulation layer 30, and the antenna array 50 is integrated into the touch layer 60.

[0100] Figure 11 This is a plan view of a second display panel provided in an embodiment of this disclosure. Specifically, as shown... Figure 11 As shown, the second display panel 10 may have a display area 10a, an opening area 10b, and a transition area 10c located between the display area 10a and the opening area 10b, the transition area 10c being disposed around the opening area 10b; it should be noted that the second display panel 10 may also include an isolation area 10d, an inner ring wiring area 10e, an outer ring wiring area 10f, and an outer ring encapsulation area 10g; the isolation area 10d is located between the display area 10a and the transition area 10c and is disposed around the transition area 10c; the inner ring wiring area 10e is located between the isolation area 10d and the display area 10a and is disposed around the isolation area 10d; the outer ring wiring area 10f may be disposed around the display area 10a or disposed on both sides of the display area 10a; and the outer ring encapsulation area 10g may encapsulate the entire second display panel 10 at the outermost edge of the entire substrate.

[0101] Specifically, the second display panel may include a substrate, a driving circuit layer, a display device (OLED layer), and a thin film encapsulation layer; wherein, the substrate may be a flexible substrate to improve the flexibility of the second display panel 10, so that the second display panel 10 can have properties such as bendability and flexibility, so as to expand the application range of the second display panel 10; but not limited thereto, the substrate may also be set to rigid, and the specific performance of the substrate may be determined according to the actual needs of the product.

[0102] Furthermore, the substrate can be a single-layer structure or a multi-layer structure.

[0103] Figure 12 for Figure 11 The second display panel is shown in a cross-sectional view along the AA direction. Figure 12As shown, in some embodiments, the substrate may include a polyimide layer 101 and a buffer layer 102 stacked sequentially. In other embodiments, the substrate may include multiple polyimide layers 101 and buffer layers 102 stacked sequentially. The buffer layer 102 may be made of materials such as silicon nitride and silicon oxide to achieve the effects of blocking water and oxygen and blocking alkaline ions. It should be noted that the structure of the substrate is not limited to this and can be determined according to actual needs.

[0104] It should be noted that, in order to facilitate the subsequent processing of the required components in each area of ​​the second display panel 10, each area can be defined in advance on the substrate. For example, the display area 10a, transition area 10c, aperture area 10b, isolation area 10d, inner ring wiring area 10e, outer ring wiring area 10f, and outer ring encapsulation area 10g can be defined on the substrate, that is, the entire second display panel 10 is divided into the display area 10a, transition area 10c, aperture area 10b, isolation area 10d, inner ring wiring area 10e, outer ring wiring area 10f, and outer ring encapsulation area 10g.

[0105] In some embodiments, such as Figure 12 As shown, the driving circuit layer can be formed on the substrate; specifically, the driving circuit layer can be formed on the buffer layer 102 of the substrate. This driving circuit layer may include an interlayer dielectric layer 103 located between the display area 10a and the transition area 10c. This interlayer dielectric layer 103 is made of inorganic materials, such as silicon oxide or silicon nitride, to achieve the effects of blocking water and oxygen and blocking alkaline ions. It should be understood that when the second display panel 10 has an isolation area 10d, an inner ring wiring area 10e, an outer ring wiring area 10f, and an outer ring encapsulation area 10g, the interlayer dielectric layer 103 is also located in the isolation area 10d, the inner ring wiring area 10e, the outer ring wiring area 10f, and the outer ring encapsulation area 10g.

[0106] Specifically, the portion of the driving circuit layer located in the display area 10a may include thin-film transistors and capacitor structures.

[0107] like Figure 12As shown, the thin-film transistor can be a top-gate type. This thin-film transistor may include an active layer 104, a first gate insulating layer 105, a gate 106, a second gate insulating layer 108, an interlayer dielectric layer 103, a source 110, and a drain 111. Specifically, the active layer 104 may be formed on a buffer layer 102. The first gate insulating layer 105 covers the buffer layer 102 and the active layer 104. The gate 106 is formed on the side of the first gate insulating layer 105 away from the active layer 104. The second gate insulating layer 108 covers the gate 106 and the first gate insulating layer 105. The interlayer dielectric layer 103 covers the second gate insulating layer 108. The source 110 and the drain 111 are formed on the side of the interlayer dielectric layer 103 away from the substrate and are located on opposite sides of the gate 106, respectively. The source 110 and the drain 111 can contact the opposite sides of the active layer 104 through vias (e.g., metal vias). It should be understood that this thin-film transistor can also be a bottom-gate type.

[0108] like Figure 12 As shown, the capacitor structure may include a first electrode 130 and a second electrode 131. The first electrode 130 is disposed on the same layer as the gate 103, and the second electrode 131 is located between the second gate insulating layer 105 and the interlayer dielectric layer 103, and is disposed opposite to the first electrode 130.

[0109] For example, the materials of the gate 103, the first electrode 130, and the second electrode 131 may include metallic or alloy materials, such as molybdenum, aluminum, and titanium. The source 110 and the drain 111 may include metallic or alloy materials, such as a single-layer or multi-layer metal structure formed of molybdenum, aluminum, and titanium. For example, the multi-layer structure is a multi-metal stack, such as a titanium, aluminum, and titanium three-layer metal stack (Al / Ti / Al).

[0110] It should be noted that the first gate insulating layer 105 and the second gate insulating layer 108 mentioned above are also located in the transition region 10c, the isolation region 10d, the inner ring routing region 10e, the outer ring routing region 10f, and the outer ring encapsulation region 10g.

[0111] like Figure 12 As shown, the display device is located in the display area. The display device may include a first electrode 112 and a pixel defining portion 113 sequentially formed on the interlayer dielectric layer 103. It should be understood that the display device may also include a light-emitting portion 114a and a second electrode 115.

[0112] Specifically, when the thin-film transistor in display area 10a is a top-gate type, a planarization layer can be fabricated before fabricating the display device. This planarization layer can be a single-layer structure or a multi-layer structure; this planarization layer is usually made of organic materials, such as photoresist, acrylic polymers, silicon polymers, etc. Figure 12As shown, this planarization layer may include a planarization portion 116 located in the display area 10a, which is formed between the interlayer dielectric layer 103 and the first electrode 112. The first electrode 112 can be electrically connected to the drain electrode 111 via a metal via. The first electrode 112 can be an anode, made of materials such as ITO (indium tin oxide), indium zinc oxide (IZO), or zinc oxide (ZnO). A pixel defining portion 113 may cover the planarization portion 116. This pixel defining portion 113 can be made of organic materials, such as photoresist, and the portion of the pixel defining portion 113 located in the display area 10a may have a pixel opening exposing the first electrode 112. A light-emitting portion 114a is located within the pixel opening and formed on the first electrode 112. The light-emitting portion 114a may include small... The molecular organic material or polymeric molecular organic material can be a fluorescent luminescent material or a phosphorescent luminescent material, which can emit red, green, blue, or white light, etc.; and, depending on different actual needs, in different examples, the light-emitting part 114a can further include functional layers such as an electron injection layer, an electron transport layer, a hole injection layer, and a hole transport layer; the second electrode 115 covers the light-emitting part 114a, and the polarity of the second electrode 115 is opposite to that of the first electrode 112; this second electrode 115 can be a cathode, which can be made of metal materials such as lithium (Li), aluminum (Al), magnesium (Mg), and silver (Ag).

[0113] It should be noted that, as Figure 12 As shown, the first electrode 112, the light-emitting part 114a, and the second electrode 115 can constitute a light-emitting sub-pixel 1d. The portion of the display area 10a in this display device may include multiple light-emitting sub-pixels 1d arranged in an array. Furthermore, it should be noted that the first electrodes 112 of each light-emitting sub-pixel 1d are independent of each other, and the second electrodes 115 of each light-emitting sub-pixel 1d are connected across their entire surface; that is, the second electrode 115 is a full-surface structure disposed on the second display panel 10, serving as a common electrode for multiple display devices.

[0114] In some embodiments, such as Figure 12 As shown, a support portion 132 may be provided on the side of the pixel defining portion 113 facing away from the interlayer dielectric layer 103. This support portion 132 can support the protective film layer (not shown in the figure) to prevent the protective film layer from contacting the first electrode 112 or other traces, which could easily damage the first electrode 112 or other traces. It should be noted that this protective film layer mainly appears during the transfer of semi-finished products to prevent damage to the semi-finished products during the transfer process. Specifically, when transferring the substrate with the support portion 132 to the evaporation line, a protective film layer can be applied. When the evaporation of the light-emitting material is required, the protective film layer is removed.

[0115] For example, the material of the support portion 132 may be the same as the material of the pixel defining portion 113, and the support portion 132 and the pixel defining portion 113 may be formed using the same patterning process. However, it is not limited to this. The material of the support portion 132 may also be different from the material of the pixel defining portion 113, and the support portion 132 and the pixel defining portion 113 may also be formed using different patterning processes.

[0116] like Figure 12 As shown, the thin-film encapsulation layer 118 may include a first inorganic encapsulation thin-film layer 118a, an organic encapsulation thin-film layer 118b, and a second inorganic encapsulation thin-film layer 118c, which are stacked sequentially. The first inorganic encapsulation thin-film layer 118a encapsulates the display device; the organic encapsulation thin-film layer 118b encapsulates the display device; and the second inorganic encapsulation thin-film layer 118c encapsulates the display device. The first inorganic encapsulation thin-film layer 118a and the second inorganic encapsulation thin-film layer 118c prevent water and oxygen from entering the light-emitting portion 114a of the display area 10a from the display side and the opening area 10b. The first inorganic encapsulation thin-film layer 118a and the second inorganic encapsulation thin-film layer 118c may be made of inorganic materials such as silicon nitride and silicon oxide. The organic encapsulation thin-film layer 118b is used to achieve planarization to facilitate the fabrication of the second inorganic encapsulation thin-film layer 118c. This organic encapsulation thin-film layer 118b may be made of materials such as acrylic polymers and silicon polymers.

[0117] The first inorganic encapsulation film layer 118a and the second inorganic encapsulation film layer 118c can be manufactured using chemical vapor deposition (CVD), but are not limited to this; physical vapor deposition (PVD) or other processes can also be used. The organic encapsulation film layer 118b is manufactured using inkjet printing, but is not limited to this; spraying or other processes can also be used. During the fabrication of the organic encapsulation film layer 118b, because the organic encapsulation material has a certain degree of fluidity, the first barrier portion 117 can restrict the flow of the organic encapsulation material, preventing it from flowing into the opening area 10b and causing encapsulation failure.

[0118] Therefore, embodiments of this disclosure provide various integration solutions for millimeter-wave radar, including: integrating the antenna array in a display and integrating the data processing module in a printed circuit board, with the two connected by a flexible cable; or integrating the data processing module in a printed circuit board and encapsulating the antenna array in the data processing module. Furthermore, integrating the antenna array into the display includes various solutions such as integration outside the display screen, on the screen, and under the screen. Different configuration schemes are proposed for displays of different specifications and those using different display devices. For example, for a liquid crystal display (LCD), it includes a first display panel, a first polarizer located on the display surface side of the first display panel, and a second polarizer located on the side of the first display panel opposite to the first polarizer. The first display panel includes a color filter layer, a liquid crystal layer, and a backplate stacked sequentially. The antenna array can be attached to the side of the first polarizer opposite to the display surface of the first display panel, or the antenna array can be integrated between the first polarizer and the color filter layer. The antenna array can also be simultaneously disposed in the display with the touch layer. Similarly, for an OLED display, it includes a second display panel and a third polarizer located on the display surface side of the second display panel. The second display panel includes a thin-film encapsulation layer and an OLED layer stacked together. The antenna array can be attached to the side of the third polarizer opposite to the display surface of the second display panel, or the antenna array can be integrated between the third polarizer and the thin-film encapsulation layer. Thus, through the efficient and integrated design of millimeter-wave radar, display modules, and terminal devices, non-contact control is achieved, enhancing the flexibility of interactive control.

[0119] This disclosure also provides an interactive system, which includes: a terminal device and an interactive control device as described in any of the above embodiments; the terminal device is used to operate according to control instructions output by the interactive control device, and in some embodiments, the terminal device includes a device processor, a device actuator, and other necessary device components.

[0120] The interactive control device and interactive system provided in this disclosure embodiment will be described in detail below with reference to practical applications. Taking a smart home scenario as an example, the terminal devices are the corresponding smart home devices. The interactive control device provided in this disclosure embodiment can realize contactless control of one or more smart home devices. The terminal devices may include: home televisions, air conditioners, lights, electronic curtains, water heaters, range hoods, smart stoves, refrigerators, audio equipment, electronic doors, etc. The interactive control device can be set up independently or installed inside the corresponding terminal device. It can establish a one-to-one control, one-to-many control, or many-to-one control relationship with each terminal device.

[0121] Taking a range hood as the terminal device and an interactive control device set inside the range hood for one-to-one control as an example.

[0122] The interactive control device includes an antenna array, a data processing module, and a display module. The antenna array is used to transmit radar signals and receive reflected echo signals. Specifically, the radar signal is a frequency-modulated continuous wave signal. The data processing module includes a low-noise amplifier, a mixer, an intermediate frequency amplifier, an analog-to-digital converter, a digital front-end component, a buffer, a power amplifier, a power divider, a waveform generator, a first processing core, and a second processing core. The display module is used to display determined human actions and / or control commands, as well as to display the range hood's interactive interface. The interactive control device shares the display module with the range hood, and the antenna array is integrated into the display module's display, including integration within the screen and integration outside the screen. Integration within the screen includes integration on the screen and integration under the screen.

[0123] First, the waveform generator in the data processing module generates radar signals and sends them to the mixer and power amplifier via the power divider. After processing by the power amplifier, the signals are sent to the antenna array for transmission. The low-noise amplifier receives the echo signals received by the antenna array. The mixer mixes the radar signals with the echo signals. After a series of processing steps by the intermediate frequency amplifier, analog-to-digital converter, and digital front-end components, the corresponding data is stored in the buffer. The first processing core analyzes and processes this data, including one-dimensional Fourier transform, two-dimensional Fourier transform, and angle of arrival calculation, to generate information about the reflecting object. The second processing core recognizes hand gestures based on the information about the reflecting object using a pre-trained gesture recognition network, determines the user's desired control command for the range hood, and sends the control command to the range hood through the corresponding interface. The range hood then executes the corresponding action. Specifically, for example, when a hand gesture is detected as staying in the designated sensing area for more than 3 seconds, the range hood is turned on; when a hand gesture is detected as clockwise or counterclockwise rotation, the range hood's fan speed is adjusted; and when a hand gesture is detected as waving left or right, the display interface is turned on.

[0124] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. An interactive control device, comprising: Antenna arrays are used to transmit radar signals and receive reflected echo signals. The data processing module is used to determine human body movements and corresponding control commands based on the radar signal and the echo signal, and output the control commands to the terminal device; specifically, the data processing module is used to mix the radar signal and the echo signal and perform analysis processing to obtain information about the reflecting object, and to identify the human body movements based on the information about the reflecting object and determine the corresponding control commands. A display module is used to display the determined human body movements and / or the control commands; The data processing module includes: a first processing core and a second processing core; The first processing core is used to perform analysis and calculation on the signal obtained after mixing the radar signal and the echo signal to generate the information of the reflecting object. The analysis and calculation includes at least one of one-dimensional Fourier transform, two-dimensional Fourier transform, and angle of arrival calculation. The second processing core is used to perform chirp control on the radar signal, pre-train and generate a human motion recognition network, and identify the human motion through the human motion recognition network based on the information of the reflecting object, and determine the corresponding control command. The display module includes a display, the display includes a display panel and a polarizer located on the display surface side of the display panel, and the antenna array is located between the display panel and the polarizer.

2. The interactive control device according to claim 1, wherein, The information about the reflecting object includes at least one of distance information, speed information, and angle of arrival information.

3. The interactive control device according to claim 2, wherein, The radar signal is a frequency-modulated continuous wave signal.

4. The interactive control device according to claim 1, wherein, The data processing module includes: a low-noise amplifier, a mixer, an intermediate frequency amplifier, an analog-to-digital converter, a digital front-end component, a buffer, a power amplifier, a power divider, and a waveform generator; The low-noise amplifier, the mixer, the intermediate frequency amplifier, the analog-to-digital converter, the digital front-end component, and the buffer are connected in sequence. The waveform generator, the power divider, and the power amplifier are connected in sequence; The input of the mixer is also connected to the power divider, the input of the low-noise amplifier is connected to the antenna array, and the output of the power amplifier is connected to the antenna array.

5. The interactive control device according to claim 1, wherein, The antenna array is integrated into the display, and the antenna array includes a reference electrode layer, a dielectric layer and a radiating electrode layer stacked in sequence.

6. The interactive control device according to claim 5, wherein, The display has a display area and a peripheral area surrounding the display area; the antenna array is integrated in the peripheral area, and both the reference electrode layer and the radiating electrode layer include planar electrodes.

7. The interactive control device according to claim 5, wherein, The display has a display area and a peripheral area surrounding the display area; the antenna array is integrated in the display area, and both the reference electrode layer and the radiating electrode layer include a metal mesh.

8. The interactive control device according to claim 7, wherein, The display is a liquid crystal display, and the display includes a first display panel, a first polarizer located on the display surface side of the first display panel, and a second polarizer located on the side of the first display panel opposite to the first polarizer.

9. The interactive control device according to claim 8, wherein, The first display panel includes a color filter layer, a liquid crystal layer, and a back panel stacked sequentially. The antenna array is integrated between the first polarizer and the color filter layer.

10. The interactive control device according to claim 9, wherein, The display further includes a touch layer; the touch layer is disposed between the color filter layer and the antenna array, or the touch layer is disposed between the color filter layer and the liquid crystal layer.

11. The interactive control device according to claim 7, wherein, The display is an OLED display, and the display includes a second display panel and a third polarizer located on the display surface side of the second display panel.

12. The interactive control device according to claim 11, wherein, The second display panel includes a thin-film encapsulation layer and an OLED layer stacked together; The antenna array is integrated between the third polarizer and the thin-film encapsulation layer.

13. The interactive control device according to claim 12, wherein, The display also includes a touch layer; the touch layer is disposed between the thin film encapsulation layer and the antenna array, or the touch layer is disposed between the third polarizer and the thin film encapsulation layer, and the antenna array is integrated into the touch layer.

14. The interactive control device according to claim 5, wherein, The interactive control device also includes: a printed circuit board; The data processing module is mounted on the printed circuit board, and the antenna array transmits signals to the data processing module via a flexible cable.

15. The interactive control device according to claim 1, wherein, The interactive control device also includes: a printed circuit board; The data processing module is mounted on the printed circuit board, and the antenna array is encapsulated inside the data processing module.

16. The interactive control device according to claim 1, wherein, The display module includes: a routing gateway unit; The routing gateway unit is used to receive media data; The display module is also used to display the media data.

17. An interactive system comprising: Terminal equipment and interactive control devices; The interactive control device described herein is an interactive control device as described in any one of claims 1 to 16; The terminal device is used to operate according to the control commands output by the interactive control device.

Images